BasicOptimizer.scala:89 executed in 7.84 seconds (0.000 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 139083136116400
Reset training subject: 139083267815400
Adding measurement 243f4fd0 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.08781588915902792;dx=-3.3577956338304636E-5
New Minimum: -0.08781588915902792 > -0.08789427886608907
WOLFE (weak): th(2.154434690031884)=-0.08789427886608907; dx=-3.280378681066356E-5 evalInputDelta=7.83897070611439E-5
New Minimum: -0.08789427886608907 > -0.08797329887902959
END: th(4.308869380063768)=-0.08797329887902959; dx=-2.9068010939386177E-5 evalInputDelta=1.5740972000166664E-4
Fitness changed from -0.08781588915902792 to -0.08797329887902959
Iteration 1 complete. Error: -0.08797329887902959 Total: 0.7125; Orientation: 0.0016; Line Search: 0.3485
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 465077b0 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-0.08797329887902959;dx=-3.5875193269217556E-5
New Minimum: -0.08797329887902959 > -0.08833835920584647
WOLFE (weak): th(9.283177667225559)=-0.08833835920584647; dx=-3.404033864488272E-5 evalInputDelta=3.650603268168795E-4
New Minimum: -0.08833835920584647 > -0.08869995139331041
WOLFE (weak): th(18.566355334451117)=-0.08869995139331041; dx=-3.437541527798978E-5 evalInputDelta=7.266525142808206E-4
New Minimum: -0.08869995139331041 > -0.09012595511116805
WOLFE (weak): th(55.69906600335335)=-0.09012595511116805; dx=-3.626322383978799E-5 evalInputDelta=0.002152656232138464
New Minimum: -0.09012595511116805 > -0.09653923632954149
WOLFE (weak): th(222.7962640134134)=-0.09653923632954149; dx=-3.5216638138667576E-5 evalInputDelta=0.008565937450511901
New Minimum: -0.09653923632954149 > -0.12968986920611392
END: th(1113.981320067067)=-0.12968986920611392; dx=-3.1501962181107916E-5 evalInputDelta=0.041716570327084326
Fitness changed from -0.08797329887902959 to -0.12968986920611392
Iteration 2 complete. Error: -0.12968986920611392 Total: 0.8460; Orientation: 0.0017; Line Search: 0.7175
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 6336e41a to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-0.12968986920611392;dx=-3.0698352585071596E-5
New Minimum: -0.12968986920611392 > -0.2057785959922221
END: th(2400.0)=-0.2057785959922221; dx=-2.5354277222493965E-5 evalInputDelta=0.07608872678610817
Fitness changed from -0.12968986920611392 to -0.2057785959922221
Iteration 3 complete. Error: -0.2057785959922221 Total: 0.3604; Orientation: 0.0017; Line Search: 0.2347
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 7335c59f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.096e+02, gradient 5.616e-03, dot -0.881; [c90e5762-595d-4d33-86c6-024967c7bea7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.2057785959922221, -0.12968986920611392, -0.08797329887902959, -0.08781588915902792
LBFGS Accumulation History: 3 points
Removed measurement 7335c59f to history. Total: 3
th(0)=-0.2057785959922221;dx=-2.937468998708874E-5
New Minimum: -0.2057785959922221 > -0.3687704046610745
WOLFE (weak): th(5170.6432560765215)=-0.3687704046610745; dx=-2.793016850248539E-5 evalInputDelta=0.16299180866885243
New Minimum: -0.3687704046610745 > -0.5357332248303858
END: th(10341.286512153043)=-0.5357332248303858; dx=-1.8236066073410706E-5 evalInputDelta=0.3299546288381637
Fitness changed from -0.2057785959922221 to -0.5357332248303858
Iteration 4 complete. Error: -0.5357332248303858 Total: 0.5106; Orientation: 0.0280; Line Search: 0.3583
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 681e3e39 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.517e+02, gradient 4.667e-03, dot -0.710; [c90e5762-595d-4d33-86c6-024967c7bea7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.5357332248303858, -0.12968986920611392, -0.08797329887902959, -0.08781588915902792
LBFGS Accumulation History: 3 points
Removed measurement 681e3e39 to history. Total: 3
th(0)=-0.5357332248303858;dx=-1.615077382761406E-5
New Minimum: -0.5357332248303858 > -0.9796614878769666
END: th(22279.626401341342)=-0.9796614878769666; dx=-8.582888791421917E-6 evalInputDelta=0.44392826304658084
Fitness changed from -0.5357332248303858 to -0.9796614878769666
Iteration 5 complete. Error: -0.9796614878769666 Total: 0.3957; Orientation: 0.0279; Line Search: 0.2409
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement d3ff0d4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.815e+02, gradient 4.270e-03, dot -0.436; [c90e5762-595d-4d33-86c6-024967c7bea7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.9796614878769666, -0.12968986920611392, -0.08797329887902959, -0.08781588915902792
LBFGS Accumulation History: 3 points
Removed measurement d3ff0d4 to history. Total: 3
th(0)=-0.9796614878769666;dx=-1.4725308611641471E-5
New Minimum: -0.9796614878769666 > -1.2390774364948793
END: th(48000.00000000001)=-1.2390774364948793; dx=-4.2786054709175465E-6 evalInputDelta=0.25941594861791273
Fitness changed from -0.9796614878769666 to -1.2390774364948793
Iteration 6 complete. Error: -1.2390774364948793 Total: 0.3881; Orientation: 0.0274; Line Search: 0.2350
Adding measurement 7b826681 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.691e+02, gradient 4.233e-03, dot -0.184; [c90e5762-595d-4d33-86c6-024967c7bea7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.2390774364948793, -0.12968986920611392, -0.08797329887902959, -0.08781588915902792
LBFGS Accumulation History: 3 points
Removed measurement 7b826681 to history. Total: 3
th(0)=-1.2390774364948793;dx=-1.5157081353324453E-5
New Minimum: -1.2390774364948793 > -1.5198166374375208
END: th(103412.86512153044)=-1.5198166374375208; dx=-2.727957989376576E-7 evalInputDelta=0.2807392009426415
Fitness changed from -1.2390774364948793 to -1.5198166374375208
Iteration 7 complete. Error: -1.5198166374375208 Total: 0.3767; Orientation: 0.0277; Line Search: 0.2347
Adding measurement 541fcd3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.556e+02, gradient 8.751e-03, dot -0.101; [c90e5762-595d-4d33-86c6-024967c7bea7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.5198166374375208, -0.12968986920611392, -0.08797329887902959, -0.08781588915902792
LBFGS Accumulation History: 3 points
Removed measurement 541fcd3 to history. Total: 3
th(0)=-1.5198166374375208;dx=-5.478809707667093E-5
New Minimum: -1.5198166374375208 > -3.2100655261118707
END: th(222796.26401341346)=-3.2100655261118707; dx=-3.7367023481860226E-6 evalInputDelta=1.6902488886743499
Fitness changed from -1.5198166374375208 to -3.2100655261118707
Iteration 8 complete. Error: -3.2100655261118707 Total: 0.3683; Orientation: 0.0276; Line Search: 0.2304
Adding measurement 62b4dadf to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.360e+03, gradient 1.922e-02, dot -0.022; [c90e5762-595d-4d33-86c6-024967c7bea7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.2100655261118707, -0.12968986920611392, -0.08797329887902959, -0.08781588915902792
LBFGS Accumulation History: 3 points
Removed measurement 62b4dadf to history. Total: 3
th(0)=-3.2100655261118707;dx=-3.5399438080786717E-4
New Minimum: -3.2100655261118707 > -9.584633584602116
END: th(480000.0000000002)=-9.584633584602116; dx=-2.8912029356340225E-6 evalInputDelta=6.3745680584902455
Fitness changed from -3.2100655261118707 to -9.584633584602116
Iteration 9 complete. Error: -9.584633584602116 Total: 0.3712; Orientation: 0.0261; Line Search: 0.2315
Adding measurement 566a052e to history. Total: 3
Accepted: LBFGS Orientation magnitude: 2.354e+04, gradient 1.131e-02, dot 0.051; [c90e5762-595d-4d33-86c6-024967c7bea7 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-9.584633584602116;dx=-13.619686570842585
Armijo: th(1034128.6512153046)=-7.397731293221796; dx=1.8632145342945165E-6 evalInputDelta=-2.1869022913803207
Armijo: th(517064.3256076523)=-7.397731293221796; dx=1.9348534304109564E-6 evalInputDelta=-2.1869022913803207
Armijo: th(172354.77520255078)=-7.397731293221796; dx=7.34411055185942E-6 evalInputDelta=-2.1869022913803207
Armijo: th(43088.693800637695)=-7.397731293221796; dx=3.798377440061636E-5 evalInputDelta=-2.1869022913803207
Armijo: th(8617.738760127539)=-7.397548003655557; dx=1.2236567569302417E-4 evalInputDelta=-2.187085580946559
Armijo: th(1436.2897933545898)=-7.334503303298273; dx=8.621245709626725E-4 evalInputDelta=-2.250130281303843
Armijo: th(205.18425619351282)=-7.100361314219625; dx=0.006811433845868443 evalInputDelta=-2.4842722703824913
Armijo: th(25.648032024189103)=-6.208584212739044; dx=0.061448796111752205 evalInputDelta=-3.376049371863072
Armijo: th(2.8497813360210116)=-8.32535555156841; dx=0.6046952409683146 evalInputDelta=-1.2592780330337057
New Minimum: -9.584633584602116 > -12.802790556203254
WOLF (strong): th(0.28497813360210117)=-12.802790556203254; dx=1.322262307922917 evalInputDelta=3.218156971601138
END: th(0.02590710305473647)=-10.244970573294786; dx=-7.449880254400583 evalInputDelta=0.66033698869267
Fitness changed from -9.584633584602116 to -12.802790556203254
Iteration 10 complete. Error: -12.802790556203254 Total: 1.5611; Orientation: 0.0265; Line Search: 1.4194
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 31527495 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.661e+04, gradient 1.002e-02, dot -0.012; [c90e5762-595d-4d33-86c6-024967c7bea7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.802790556203254, -9.584633584602116, -0.12968986920611392, -0.08797329887902959, -0.08781588915902792
Rejected: LBFGS Orientation magnitude: 1.670e+04, gradient 1.002e-02, dot -0.012; [c90e5762-595d-4d33-86c6-024967c7bea7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.802790556203254, -9.584633584602116, -0.12968986920611392, -0.08797329887902959
LBFGS Accumulation History: 3 points
Removed measurement 31527495 to history. Total: 4
Removed measurement 566a052e to history. Total: 3
th(0)=-12.802790556203254;dx=-9.3178498158785E-5
New Minimum: -12.802790556203254 > -12.802795811149123
END: th(0.055815161539355236)=-12.802795811149123; dx=-7.090553741732242E-5 evalInputDelta=5.254945868671257E-6
Fitness changed from -12.802790556203254 to -12.802795811149123
Iteration 11 complete. Error: -12.802795811149123 Total: 0.4210; Orientation: 0.0584; Line Search: 0.2385
Adding measurement 75d69528 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.948e+03, gradient 1.001e-02, dot -0.012; [c90e5762-595d-4d33-86c6-024967c7bea7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.802795811149123, -0.12968986920611392, -0.08797329887902959, -0.08781588915902792
LBFGS Accumulation History: 3 points
Removed measurement 75d69528 to history. Total: 3
th(0)=-12.802795811149123;dx=-9.29722039590431E-5
New Minimum: -12.802795811149123 > -12.802810363295658
END: th(0.12025012025012032)=-12.802810363295658; dx=-7.224724804941739E-5 evalInputDelta=1.4552146534896337E-5
Fitness changed from -12.802795811149123 to -12.802810363295658
Iteration 12 complete. Error: -12.802810363295658 Total: 0.3734; Orientation: 0.0281; Line Search: 0.2332
Adding measurement 50f17df8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.734e+03, gradient 1.013e-02, dot -0.021; [c90e5762-595d-4d33-86c6-024967c7bea7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.802810363295658, -0.12968986920611392, -0.08797329887902959, -0.08781588915902792
LBFGS Accumulation History: 3 points
Removed measurement 50f17df8 to history. Total: 3
th(0)=-12.802810363295658;dx=-9.295138381111516E-5
New Minimum: -12.802810363295658 > -12.802841892889742
END: th(0.2590710305473647)=-12.802841892889742; dx=-7.553026350328888E-5 evalInputDelta=3.152959408403433E-5
Fitness changed from -12.802810363295658 to -12.802841892889742
Iteration 13 complete. Error: -12.802841892889742 Total: 0.3777; Orientation: 0.0266; Line Search: 0.2373
Adding measurement 6f8cd10d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.613e+03, gradient 1.016e-02, dot -0.021; [c90e5762-595d-4d33-86c6-024967c7bea7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.802841892889742, -0.12968986920611392, -0.08797329887902959, -0.08781588915902792
LBFGS Accumulation History: 3 points
Removed measurement 6f8cd10d to history. Total: 3
th(0)=-12.802841892889742;dx=-9.540357426969515E-5
New Minimum: -12.802841892889742 > -12.802901313836758
END: th(0.5581516153935523)=-12.802901313836758; dx=-7.430760955797172E-5 evalInputDelta=5.9420947016164405E-5
Fitness changed from -12.802841892889742 to -12.802901313836758
Iteration 14 complete. Error: -12.802901313836758 Total: 0.3823; Orientation: 0.0281; Line Search: 0.2414
Adding measurement 5ab4cdb9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.736e+03, gradient 1.011e-02, dot -0.023; [c90e5762-595d-4d33-86c6-024967c7bea7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.802901313836758, -0.12968986920611392, -0.08797329887902959, -0.08781588915902792
LBFGS Accumulation History: 3 points
Removed measurement 5ab4cdb9 to history. Total: 3
th(0)=-12.802901313836758;dx=-9.46556272800471E-5
New Minimum: -12.802901313836758 > -12.80302904772802
END: th(1.202501202501203)=-12.80302904772802; dx=-7.321212968948326E-5 evalInputDelta=1.277338912615278E-4
Fitness changed from -12.802901313836758 to -12.80302904772802
Iteration 15 complete. Error: -12.80302904772802 Total: 0.3865; Orientation: 0.0286; Line Search: 0.2412
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -12.80302904772802 (> -Infinity) after 7.843s (< 5400.000s)
Returns
-12.80302904772802