BasicOptimizer.scala:89 executed in 149.77 seconds (2.766 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 461917176031300
Reset training subject: 461920300728000
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -103.20287104795639 > -103.21249193307617
th(0)=-103.21249193307617;dx=-1.529706409070722E-5
New Minimum: -103.21249193307617 > -103.21252423448148
END: th(2.154434690031884)=-103.21252423448148; dx=-1.2352966681114745E-5 evalInputDelta=3.230140531229608E-5
Fitness changed from -103.20287104795639 to -103.21252423448148
Iteration 1 complete. Error: -103.21252423448148 Total: 14.3702; Orientation: 0.0675; Line Search: 5.6856
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 73118917 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-103.21252423448148;dx=-1.2358870033955241E-5
New Minimum: -103.21252423448148 > -103.21258345369827
WOLFE (weak): th(4.641588833612779)=-103.21258345369827; dx=-1.2354759133433172E-5 evalInputDelta=5.9219216794303975E-5
New Minimum: -103.21258345369827 > -103.21264267288109
WOLFE (weak): th(9.283177667225559)=-103.21264267288109; dx=-1.2347558627239156E-5 evalInputDelta=1.1843839961045433E-4
New Minimum: -103.21264267288109 > -103.21289031636837
WOLFE (weak): th(27.849533001676676)=-103.21289031636837; dx=-1.231787019278362E-5 evalInputDelta=3.66081886895131E-4
New Minimum: -103.21289031636837 > -103.21397240376626
WOLFE (weak): th(111.3981320067067)=-103.21397240376626; dx=-1.2324064695765214E-5 evalInputDelta=0.0014481692847851946
New Minimum: -103.21397240376626 > -103.21975411134913
WOLFE (weak): th(556.9906600335335)=-103.21975411134913; dx=-1.2328558171827041E-5 evalInputDelta=0.007229876867654639
New Minimum: -103.21975411134913 > -103.25590128355934
WOLFE (weak): th(3341.9439602012008)=-103.25590128355934; dx=-1.2315584299900381E-5 evalInputDelta=0.04337704907786133
New Minimum: -103.25590128355934 > -103.51557789301954
WOLFE (weak): th(23393.607721408407)=-103.51557789301954; dx=-1.2273580112015737E-5 evalInputDelta=0.30305365853806165
New Minimum: -103.51557789301954 > -105.59266011654857
WOLFE (weak): th(187148.86177126726)=-105.59266011654857; dx=-1.1813796962844694E-5 evalInputDelta=2.380135882067094
New Minimum: -105.59266011654857 > -120.8368751055929
END: th(1684339.7559414054)=-120.8368751055929; dx=-7.617627616454022E-6 evalInputDelta=17.624350871111417
Fitness changed from -103.21252423448148 to -120.8368751055929
Iteration 2 complete. Error: -120.8368751055929 Total: 31.8434; Orientation: 0.1112; Line Search: 28.4261
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 6ce41de0 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-120.8368751055929;dx=-7.774521007407993E-6
New Minimum: -120.8368751055929 > -140.4410094369506
END: th(3628800.0000000005)=-140.4410094369506; dx=-3.281148548762589E-6 evalInputDelta=19.6041343313577
Fitness changed from -120.8368751055929 to -140.4410094369506
Iteration 3 complete. Error: -140.4410094369506 Total: 9.3198; Orientation: 0.1047; Line Search: 5.9089
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 7faa9b95 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-140.4410094369506;dx=-5.046860263132322E-6
New Minimum: -140.4410094369506 > -158.15578647108688
END: th(7818012.603187701)=-158.15578647108688; dx=-6.669977076911665E-7 evalInputDelta=17.71477703413629
Fitness changed from -140.4410094369506 to -158.15578647108688
Iteration 4 complete. Error: -158.15578647108688 Total: 9.3291; Orientation: 0.1068; Line Search: 5.9360
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 39ee7935 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.717e+04, gradient 2.492e-03, dot -0.483; [00e65955-0850-418e-b2b4-99880623af50 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -158.15578647108688, -140.4410094369506, -120.8368751055929, -103.21252423448148
LBFGS Accumulation History: 3 points
Removed measurement 39ee7935 to history. Total: 3
th(0)=-158.15578647108688;dx=-6.2081032659989985E-6
Armijo: th(1.6843397559414055E7)=-152.15652095125367; dx=1.7486003224852247E-6 evalInputDelta=-5.999265519833216
New Minimum: -158.15578647108688 > -169.11153702836845
WOLF (strong): th(8421698.779707028)=-169.11153702836845; dx=1.3137643692358439E-6 evalInputDelta=10.955750557281561
New Minimum: -169.11153702836845 > -170.38402997033918
END: th(2807232.926569009)=-170.38402997033918; dx=-2.254935621637177E-6 evalInputDelta=12.2282434992523
Fitness changed from -158.15578647108688 to -170.38402997033918
Iteration 5 complete. Error: -170.38402997033918 Total: 16.9104; Orientation: 1.8746; Line Search: 11.5793
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 7b8b6012 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.835e+04, gradient 1.754e-03, dot -0.648; [00e65955-0850-418e-b2b4-99880623af50 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -170.38402997033918, -140.4410094369506, -120.8368751055929, -103.21252423448148
LBFGS Accumulation History: 3 points
Removed measurement 7b8b6012 to history. Total: 3
th(0)=-170.38402997033918;dx=-3.0711279110997685E-6
New Minimum: -170.38402997033918 > -181.39872017521355
END: th(6048000.000000001)=-181.39872017521355; dx=-9.248319648461721E-7 evalInputDelta=11.01469020487437
Fitness changed from -170.38402997033918 to -181.39872017521355
Iteration 6 complete. Error: -181.39872017521355 Total: 10.9220; Orientation: 1.9409; Line Search: 5.7774
Adding measurement 4d395afb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.669e+04, gradient 1.915e-03, dot -0.563; [00e65955-0850-418e-b2b4-99880623af50 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -181.39872017521355, -140.4410094369506, -120.8368751055929, -103.21252423448148
LBFGS Accumulation History: 3 points
Removed measurement 4d395afb to history. Total: 3
th(0)=-181.39872017521355;dx=-3.6665352401445596E-6
New Minimum: -181.39872017521355 > -183.95362696687056
WOLF (strong): th(1.3030021005312836E7)=-183.95362696687056; dx=8.828954140117023E-7 evalInputDelta=2.554906791657004
New Minimum: -183.95362696687056 > -189.3773595672452
WOLF (strong): th(6515010.502656418)=-189.3773595672452; dx=2.012496457164154E-7 evalInputDelta=7.978639392031653
END: th(2171670.167552139)=-187.38545315092986; dx=-1.7390471990965749E-6 evalInputDelta=5.986732975716308
Fitness changed from -181.39872017521355 to -189.3773595672452
Iteration 7 complete. Error: -189.3773595672452 Total: 16.2847; Orientation: 1.7255; Line Search: 11.5671
Adding measurement 44014fdb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.967e+04, gradient 2.125e-03, dot -0.470; [00e65955-0850-418e-b2b4-99880623af50 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -189.3773595672452, -140.4410094369506, -120.8368751055929, -103.21252423448148
LBFGS Accumulation History: 3 points
Removed measurement 44014fdb to history. Total: 3
th(0)=-189.3773595672452;dx=-4.5117389718348394E-6
New Minimum: -189.3773595672452 > -199.85188827929616
END: th(4678721.544281682)=-199.85188827929616; dx=-4.389042135665257E-8 evalInputDelta=10.474528712050954
Fitness changed from -189.3773595672452 to -199.85188827929616
Iteration 8 complete. Error: -199.85188827929616 Total: 10.7013; Orientation: 1.9236; Line Search: 5.9599
Adding measurement 7240ab6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.966e+04, gradient 1.713e-03, dot -0.557; [00e65955-0850-418e-b2b4-99880623af50 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -199.85188827929616, -140.4410094369506, -120.8368751055929, -103.21252423448148
LBFGS Accumulation History: 3 points
Removed measurement 7240ab6 to history. Total: 3
th(0)=-199.85188827929616;dx=-2.93275325373609E-6
New Minimum: -199.85188827929616 > -204.30184378973865
WOLF (strong): th(1.0080000000000002E7)=-204.30184378973865; dx=7.722012386358613E-7 evalInputDelta=4.449955510442493
New Minimum: -204.30184378973865 > -207.06436910357394
END: th(5040000.000000001)=-207.06436910357394; dx=-2.1357255621552386E-7 evalInputDelta=7.212480824277776
Fitness changed from -199.85188827929616 to -207.06436910357394
Iteration 9 complete. Error: -207.06436910357394 Total: 13.2690; Orientation: 1.7932; Line Search: 8.6965
Adding measurement 23d6a69 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.562e+04, gradient 1.679e-03, dot -0.548; [00e65955-0850-418e-b2b4-99880623af50 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -207.06436910357394, -140.4410094369506, -120.8368751055929, -103.21252423448148
LBFGS Accumulation History: 3 points
Removed measurement 23d6a69 to history. Total: 3
th(0)=-207.06436910357394;dx=-2.815634322770076E-6
New Minimum: -207.06436910357394 > -209.54117077611124
WOLF (strong): th(1.0858350837760696E7)=-209.54117077611124; dx=8.850225708697114E-7 evalInputDelta=2.476801672537306
New Minimum: -209.54117077611124 > -213.54720384873838
WOLF (strong): th(5429175.418880348)=-213.54720384873838; dx=4.501965668985022E-8 evalInputDelta=6.482834745164439
END: th(1809725.1396267826)=-211.2191009655953; dx=-1.662456005457539E-6 evalInputDelta=4.15473186202135
Fitness changed from -207.06436910357394 to -213.54720384873838
Iteration 10 complete. Error: -213.54720384873838 Total: 16.0055; Orientation: 1.8102; Line Search: 11.5165
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -213.54720384873838 (> -Infinity) after 149.768s (< 5400.000s)
Returns
-213.54720384873838