BasicOptimizer.scala:89 executed in 134.88 seconds (1.700 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 74303978966900
Reset training subject: 74306660669300
Adding measurement 30b1d923 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-20.357278753415528;dx=-7.184872859206445E-7
New Minimum: -20.357278753415528 > -20.35727938743243
END: th(2.154434690031884)=-20.35727938743243; dx=-5.92372947597261E-7 evalInputDelta=6.340169029783738E-7
Fitness changed from -20.357278753415528 to -20.35727938743243
Iteration 1 complete. Error: -20.35727938743243 Total: 13.2015; Orientation: 0.0596; Line Search: 5.3588
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 5a8c3773 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-20.35727938743243;dx=-5.900504043506791E-7
New Minimum: -20.35727938743243 > -20.357282557516644
WOLFE (weak): th(4.641588833612779)=-20.357282557516644; dx=-5.897527857371543E-7 evalInputDelta=3.170084212911206E-6
New Minimum: -20.357282557516644 > -20.357285727600363
WOLFE (weak): th(9.283177667225559)=-20.357285727600363; dx=-5.900374868558058E-7 evalInputDelta=6.340167931995211E-6
New Minimum: -20.357285727600363 > -20.357298407930305
WOLFE (weak): th(27.849533001676676)=-20.357298407930305; dx=-5.89812596264341E-7 evalInputDelta=1.902049787361193E-5
New Minimum: -20.357298407930305 > -20.35735229924444
WOLFE (weak): th(111.3981320067067)=-20.35735229924444; dx=-5.898555290069707E-7 evalInputDelta=7.291181200841379E-5
New Minimum: -20.35735229924444 > -20.35764330987556
WOLFE (weak): th(556.9906600335335)=-20.35764330987556; dx=-5.897034882562538E-7 evalInputDelta=3.639224431282173E-4
New Minimum: -20.35764330987556 > -20.359465360310725
WOLFE (weak): th(3341.9439602012008)=-20.359465360310725; dx=-5.892995284719958E-7 evalInputDelta=0.002185972878294251
New Minimum: -20.359465360310725 > -20.372562334035504
WOLFE (weak): th(23393.607721408407)=-20.372562334035504; dx=-5.879691530611567E-7 evalInputDelta=0.01528294660307239
New Minimum: -20.372562334035504 > -20.47823663661055
WOLFE (weak): th(187148.86177126726)=-20.47823663661055; dx=-5.743402662465627E-7 evalInputDelta=0.12095724917811879
New Minimum: -20.47823663661055 > -21.239947106347675
END: th(1684339.7559414054)=-21.239947106347675; dx=-3.1827466110760333E-7 evalInputDelta=0.8826677189152434
Fitness changed from -20.35727938743243 to -21.239947106347675
Iteration 2 complete. Error: -21.239947106347675 Total: 28.9915; Orientation: 0.0584; Line Search: 25.7835
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 7d55490 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-21.239947106347675;dx=-4.948117268612522E-7
New Minimum: -21.239947106347675 > -22.208515638650358
END: th(3628800.0000000005)=-22.208515638650358; dx=-1.17368631070831E-7 evalInputDelta=0.9685685323026831
Fitness changed from -21.239947106347675 to -22.208515638650358
Iteration 3 complete. Error: -22.208515638650358 Total: 8.3573; Orientation: 0.0596; Line Search: 5.2302
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement dd921af to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.182e+03, gradient 7.370e-04, dot -0.472; [6cbd5ca8-9962-464e-9bdd-c2f0d16afbbd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -22.208515638650358, -21.239947106347675, -20.35727938743243, -20.357278753415528
LBFGS Accumulation History: 3 points
Removed measurement dd921af to history. Total: 3
th(0)=-22.208515638650358;dx=-5.430569276613456E-7
New Minimum: -22.208515638650358 > -23.00296180457559
END: th(7818012.603187701)=-23.00296180457559; dx=-2.015583651768043E-8 evalInputDelta=0.7944461659252333
Fitness changed from -22.208515638650358 to -23.00296180457559
Iteration 4 complete. Error: -23.00296180457559 Total: 9.4330; Orientation: 1.0722; Line Search: 5.3079
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 153eec0e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.052e+03, gradient 6.656e-04, dot -0.231; [6cbd5ca8-9962-464e-9bdd-c2f0d16afbbd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -23.00296180457559, -21.239947106347675, -20.35727938743243, -20.357278753415528
LBFGS Accumulation History: 3 points
Removed measurement 153eec0e to history. Total: 3
th(0)=-23.00296180457559;dx=-4.427762144816041E-7
New Minimum: -23.00296180457559 > -24.716702877525226
WOLF (strong): th(1.6843397559414055E7)=-24.716702877525226; dx=7.705571778589142E-9 evalInputDelta=1.7137410729496345
New Minimum: -24.716702877525226 > -24.813713277277184
END: th(8421698.779707028)=-24.813713277277184; dx=-8.586923521413736E-9 evalInputDelta=1.8107514727015932
Fitness changed from -23.00296180457559 to -24.813713277277184
Iteration 5 complete. Error: -24.813713277277184 Total: 12.0130; Orientation: 1.0149; Line Search: 7.8631
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 2bb1e4d7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.552e+03, gradient 7.087e-04, dot -0.291; [6cbd5ca8-9962-464e-9bdd-c2f0d16afbbd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -24.813713277277184, -21.239947106347675, -20.35727938743243, -20.357278753415528
LBFGS Accumulation History: 3 points
Removed measurement 2bb1e4d7 to history. Total: 3
th(0)=-24.813713277277184;dx=-5.020663536085296E-7
New Minimum: -24.813713277277184 > -25.369047022106045
WOLF (strong): th(1.8144000000000004E7)=-25.369047022106045; dx=4.2131068987286705E-8 evalInputDelta=0.5553337448288609
New Minimum: -25.369047022106045 > -26.148893975351974
WOLF (strong): th(9072000.000000002)=-26.148893975351974; dx=9.757113319878699E-8 evalInputDelta=1.3351806980747902
New Minimum: -26.148893975351974 > -26.192872477533786
END: th(3024000.0000000005)=-26.192872477533786; dx=-2.4192722811755827E-7 evalInputDelta=1.379159200256602
Fitness changed from -24.813713277277184 to -26.192872477533786
Iteration 6 complete. Error: -26.192872477533786 Total: 14.4861; Orientation: 0.9797; Line Search: 10.3892
Adding measurement 3a04b334 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.335e+04, gradient 6.480e-04, dot -0.444; [6cbd5ca8-9962-464e-9bdd-c2f0d16afbbd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -26.192872477533786, -21.239947106347675, -20.35727938743243, -20.357278753415528
LBFGS Accumulation History: 3 points
Removed measurement 3a04b334 to history. Total: 3
th(0)=-26.192872477533786;dx=-4.1943379845766466E-7
New Minimum: -26.192872477533786 > -26.936067789719512
WOLF (strong): th(6515010.502656418)=-26.936067789719512; dx=5.533024383985647E-8 evalInputDelta=0.7431953121857262
New Minimum: -26.936067789719512 > -27.009063800888644
END: th(3257505.251328209)=-27.009063800888644; dx=-5.334344957030282E-8 evalInputDelta=0.8161913233548574
Fitness changed from -26.192872477533786 to -27.009063800888644
Iteration 7 complete. Error: -27.009063800888644 Total: 11.3889; Orientation: 1.0243; Line Search: 7.8632
Adding measurement 4fc7a2d2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.701e+04, gradient 6.550e-04, dot -0.458; [6cbd5ca8-9962-464e-9bdd-c2f0d16afbbd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -27.009063800888644, -21.239947106347675, -20.35727938743243, -20.357278753415528
LBFGS Accumulation History: 3 points
Removed measurement 4fc7a2d2 to history. Total: 3
th(0)=-27.009063800888644;dx=-4.287753786017239E-7
New Minimum: -27.009063800888644 > -27.241069789647288
WOLF (strong): th(7018082.316422524)=-27.241069789647288; dx=6.338311883088476E-8 evalInputDelta=0.23200598875864387
New Minimum: -27.241069789647288 > -27.59477091236729
WOLF (strong): th(3509041.158211262)=-27.59477091236729; dx=8.44234287101515E-8 evalInputDelta=0.5857071114786478
END: th(1169680.3860704207)=-27.480674028762476; dx=-2.6641165766901234E-7 evalInputDelta=0.4716102278738319
Fitness changed from -27.009063800888644 to -27.59477091236729
Iteration 8 complete. Error: -27.59477091236729 Total: 14.1210; Orientation: 0.9770; Line Search: 10.4398
Adding measurement bf5767b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.812e+04, gradient 7.227e-04, dot -0.402; [6cbd5ca8-9962-464e-9bdd-c2f0d16afbbd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -27.59477091236729, -21.239947106347675, -20.35727938743243, -20.357278753415528
LBFGS Accumulation History: 3 points
Removed measurement bf5767b to history. Total: 3
th(0)=-27.59477091236729;dx=-5.217250649969911E-7
New Minimum: -27.59477091236729 > -28.271807280536922
WOLF (strong): th(2520000.000000001)=-28.271807280536922; dx=1.027305819893822E-7 evalInputDelta=0.677036368169631
END: th(1260000.0000000005)=-28.191957415538898; dx=-2.665753234542309E-7 evalInputDelta=0.5971865031716064
Fitness changed from -27.59477091236729 to -28.271807280536922
Iteration 9 complete. Error: -28.271807280536922 Total: 11.0076; Orientation: 0.9880; Line Search: 7.5698
Adding measurement 7e986b8e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.263e+04, gradient 6.750e-04, dot -0.453; [6cbd5ca8-9962-464e-9bdd-c2f0d16afbbd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -28.271807280536922, -21.239947106347675, -20.35727938743243, -20.357278753415528
LBFGS Accumulation History: 3 points
Removed measurement 7e986b8e to history. Total: 3
th(0)=-28.271807280536922;dx=-4.5558521969344304E-7
New Minimum: -28.271807280536922 > -28.592855302086228
WOLF (strong): th(2714587.7094401745)=-28.592855302086228; dx=1.6309008163076126E-7 evalInputDelta=0.3210480215493057
New Minimum: -28.592855302086228 > -28.72048321271995
END: th(1357293.8547200873)=-28.72048321271995; dx=-7.168342459764132E-8 evalInputDelta=0.44867593218302915
Fitness changed from -28.271807280536922 to -28.72048321271995
Iteration 10 complete. Error: -28.72048321271995 Total: 11.3644; Orientation: 0.9655; Line Search: 7.7560
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -28.72048321271995 (> -Infinity) after 134.879s (< 5400.000s)
Returns
-28.72048321271995