BasicOptimizer.scala:89 executed in 295.16 seconds (3.835 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 652067041435500
Reset training subject: 652072972229000
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -34.71191709088206 > -34.755107655382794
th(0)=-34.755107655382794;dx=-1.8807234770904307E-6
New Minimum: -34.755107655382794 > -34.75511091507259
END: th(2.154434690031884)=-34.75511091507259; dx=-1.675081968430149E-6 evalInputDelta=3.259689798085219E-6
Fitness changed from -34.71191709088206 to -34.75511091507259
Iteration 1 complete. Error: -34.75511091507259 Total: 30.8777; Orientation: 0.0928; Line Search: 12.6458
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 171ede59 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-34.75511091507259;dx=-1.6789221490523182E-6
New Minimum: -34.75511091507259 > -34.755119607577235
WOLFE (weak): th(4.641588833612779)=-34.755119607577235; dx=-1.678816993529825E-6 evalInputDelta=8.692504643192933E-6
New Minimum: -34.755119607577235 > -34.75512938664235
WOLFE (weak): th(9.283177667225559)=-34.75512938664235; dx=-1.6787328537995474E-6 evalInputDelta=1.847156975998132E-5
New Minimum: -34.75512938664235 > -34.75516415662939
WOLFE (weak): th(27.849533001676676)=-34.75516415662939; dx=-1.678056162437903E-6 evalInputDelta=5.324155679886644E-5
New Minimum: -34.75516415662939 > -34.75531953458392
WOLFE (weak): th(111.3981320067067)=-34.75531953458392; dx=-1.6772952908660083E-6 evalInputDelta=2.0861951132644663E-4
New Minimum: -34.75531953458392 > -34.75614856745364
WOLFE (weak): th(556.9906600335335)=-34.75614856745364; dx=-1.6774779135118968E-6 evalInputDelta=0.001037652381050691
New Minimum: -34.75614856745364 > -34.761333105632154
WOLFE (weak): th(3341.9439602012008)=-34.761333105632154; dx=-1.6772110062468973E-6 evalInputDelta=0.00622219055956208
New Minimum: -34.761333105632154 > -34.79863637396638
WOLFE (weak): th(23393.607721408407)=-34.79863637396638; dx=-1.675218507952198E-6 evalInputDelta=0.04352545889378945
New Minimum: -34.79863637396638 > -35.10138386502358
WOLFE (weak): th(187148.86177126726)=-35.10138386502358; dx=-1.6556257527464242E-6 evalInputDelta=0.34627294995098623
New Minimum: -35.10138386502358 > -37.67603295806751
END: th(1684339.7559414054)=-37.67603295806751; dx=-1.436007773351095E-6 evalInputDelta=2.920922042994917
Fitness changed from -34.75511091507259 to -37.67603295806751
Iteration 2 complete. Error: -37.67603295806751 Total: 67.8965; Orientation: 0.1393; Line Search: 61.0106
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 48e9d155 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-37.67603295806751;dx=-1.355782626737685E-6
New Minimum: -37.67603295806751 > -42.24140647639463
END: th(3628800.0000000005)=-42.24140647639463; dx=-9.285099180295314E-7 evalInputDelta=4.565373518327121
Fitness changed from -37.67603295806751 to -42.24140647639463
Iteration 3 complete. Error: -42.24140647639463 Total: 19.9269; Orientation: 0.1484; Line Search: 12.1321
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 5a0c0f69 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-42.24140647639463;dx=-8.85692328768993E-7
New Minimum: -42.24140647639463 > -48.18535736985569
END: th(7818012.603187701)=-48.18535736985569; dx=-5.003903563654645E-7 evalInputDelta=5.943950893461057
Fitness changed from -42.24140647639463 to -48.18535736985569
Iteration 4 complete. Error: -48.18535736985569 Total: 19.5391; Orientation: 0.1389; Line Search: 12.3718
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement da550ab to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.419e+04, gradient 8.251e-04, dot -0.726; [edf04ad8-5c6d-4614-8c9a-ab04b0e45d8f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -48.18535736985569, -42.24140647639463, -37.67603295806751, -34.75511091507259
LBFGS Accumulation History: 3 points
Removed measurement da550ab to history. Total: 3
th(0)=-48.18535736985569;dx=-6.796810716909444E-7
New Minimum: -48.18535736985569 > -54.71564685981482
END: th(1.6843397559414055E7)=-54.71564685981482; dx=-1.6665179726786759E-7 evalInputDelta=6.530289489959131
Fitness changed from -48.18535736985569 to -54.71564685981482
Iteration 5 complete. Error: -54.71564685981482 Total: 22.2994; Orientation: 2.5759; Line Search: 12.3030
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 575b535d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.750e+04, gradient 9.832e-04, dot -0.481; [edf04ad8-5c6d-4614-8c9a-ab04b0e45d8f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -54.71564685981482, -42.24140647639463, -37.67603295806751, -34.75511091507259
LBFGS Accumulation History: 3 points
Removed measurement 575b535d to history. Total: 3
th(0)=-54.71564685981482;dx=-9.649338217106213E-7
Armijo: th(3.628800000000001E7)=-53.88089533810381; dx=1.6405804673461972E-7 evalInputDelta=-0.8347515217110058
New Minimum: -54.71564685981482 > -58.279278797269534
WOLF (strong): th(1.8144000000000004E7)=-58.279278797269534; dx=2.178716552580373E-7 evalInputDelta=3.5636319374547156
New Minimum: -58.279278797269534 > -59.18672964764894
END: th(6048000.000000001)=-59.18672964764894; dx=-3.3181680761004554E-7 evalInputDelta=4.471082787834121
Fitness changed from -54.71564685981482 to -59.18672964764894
Iteration 6 complete. Error: -59.18672964764894 Total: 33.5489; Orientation: 2.5269; Line Search: 24.2255
Adding measurement 37da6af5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.298e+04, gradient 7.123e-04, dot -0.676; [edf04ad8-5c6d-4614-8c9a-ab04b0e45d8f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -59.18672964764894, -42.24140647639463, -37.67603295806751, -34.75511091507259
LBFGS Accumulation History: 3 points
Removed measurement 37da6af5 to history. Total: 3
th(0)=-59.18672964764894;dx=-5.067712091373915E-7
New Minimum: -59.18672964764894 > -63.547340042388456
END: th(1.3030021005312836E7)=-63.547340042388456; dx=-1.6357119851086043E-7 evalInputDelta=4.360610394739517
Fitness changed from -59.18672964764894 to -63.547340042388456
Iteration 7 complete. Error: -63.547340042388456 Total: 20.4257; Orientation: 2.4843; Line Search: 12.1162
Adding measurement 1e627c98 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.298e+04, gradient 8.323e-04, dot -0.546; [edf04ad8-5c6d-4614-8c9a-ab04b0e45d8f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -63.547340042388456, -42.24140647639463, -37.67603295806751, -34.75511091507259
LBFGS Accumulation History: 3 points
Removed measurement 1e627c98 to history. Total: 3
th(0)=-63.547340042388456;dx=-6.919112084472722E-7
New Minimum: -63.547340042388456 > -64.57641640351058
WOLF (strong): th(2.8072329265690096E7)=-64.57641640351058; dx=1.221474443668539E-7 evalInputDelta=1.0290763611221223
New Minimum: -64.57641640351058 > -66.71891478688293
WOLF (strong): th(1.4036164632845048E7)=-66.71891478688293; dx=8.853314789863645E-8 evalInputDelta=3.171574744494478
END: th(4678721.544281683)=-66.2299484952433; dx=-3.285489367287901E-7 evalInputDelta=2.6826084528548506
Fitness changed from -63.547340042388456 to -66.71891478688293
Iteration 8 complete. Error: -66.71891478688293 Total: 32.1805; Orientation: 2.4933; Line Search: 23.8931
Adding measurement 46e2f82f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.298e+04, gradient 9.538e-04, dot -0.428; [edf04ad8-5c6d-4614-8c9a-ab04b0e45d8f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -66.71891478688293, -42.24140647639463, -37.67603295806751, -34.75511091507259
LBFGS Accumulation History: 3 points
Removed measurement 46e2f82f to history. Total: 3
th(0)=-66.71891478688293;dx=-9.093566003921373E-7
New Minimum: -66.71891478688293 > -71.78929039553239
WOLF (strong): th(1.0080000000000004E7)=-71.78929039553239; dx=4.779337204960034E-8 evalInputDelta=5.0703756086494565
END: th(5040000.000000002)=-70.70324386158727; dx=-4.782238838968672E-7 evalInputDelta=3.984329074704334
Fitness changed from -66.71891478688293 to -71.78929039553239
Iteration 9 complete. Error: -71.78929039553239 Total: 26.5032; Orientation: 2.4520; Line Search: 18.1330
Adding measurement 46112fe0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.138e+04, gradient 7.539e-04, dot -0.543; [edf04ad8-5c6d-4614-8c9a-ab04b0e45d8f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -71.78929039553239, -42.24140647639463, -37.67603295806751, -34.75511091507259
LBFGS Accumulation History: 3 points
Removed measurement 46112fe0 to history. Total: 3
th(0)=-71.78929039553239;dx=-5.67884948204252E-7
New Minimum: -71.78929039553239 > -74.97296315591223
END: th(1.0858350837760698E7)=-74.97296315591223; dx=-9.167827405527286E-9 evalInputDelta=3.183672760379835
Fitness changed from -71.78929039553239 to -74.97296315591223
Iteration 10 complete. Error: -74.97296315591223 Total: 20.7764; Orientation: 2.6525; Line Search: 12.3161
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -74.97296315591223 (> -Infinity) after 295.157s (< 5400.000s)
Returns
-74.97296315591223