BasicOptimizer.scala:89 executed in 53.31 seconds (2.137 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 219204415849800
Reset training subject: 219205044339100
Adding measurement 691c9828 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-93.3044883114553;dx=-0.08967286087916863
New Minimum: -93.3044883114553 > -93.42936298222602
WOLFE (weak): th(2.154434690031884)=-93.42936298222602; dx=-0.08963353446794281 evalInputDelta=0.12487467077072267
New Minimum: -93.42936298222602 > -93.55456646494288
WOLFE (weak): th(4.308869380063768)=-93.55456646494288; dx=-0.08976279839192944 evalInputDelta=0.25007815348757845
New Minimum: -93.55456646494288 > -94.05578696308402
WOLFE (weak): th(12.926608140191302)=-94.05578696308402; dx=-0.08987488202037966 evalInputDelta=0.7512986516287157
New Minimum: -94.05578696308402 > -96.32063305059303
WOLFE (weak): th(51.70643256076521)=-96.32063305059303; dx=-0.09048920154673777 evalInputDelta=3.0161447391377294
New Minimum: -96.32063305059303 > -108.62332746856679
WOLFE (weak): th(258.53216280382605)=-108.62332746856679; dx=-0.09069935626092636 evalInputDelta=15.318839157111483
New Minimum: -108.62332746856679 > -171.37744074497917
END: th(1551.1929768229563)=-171.37744074497917; dx=-0.06383842488655367 evalInputDelta=78.07295243352387
Fitness changed from -93.3044883114553 to -171.37744074497917
Iteration 1 complete. Error: -171.37744074497917 Total: 6.0382; Orientation: 0.0156; Line Search: 4.2782
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 5cec6d6 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-171.37744074497917;dx=-0.14064388087574334
New Minimum: -171.37744074497917 > -313.3997891616479
END: th(3341.943960201201)=-313.3997891616479; dx=-0.03962118451423195 evalInputDelta=142.02234841666873
Fitness changed from -171.37744074497917 to -313.3997891616479
Iteration 2 complete. Error: -313.3997891616479 Total: 1.7893; Orientation: 0.0164; Line Search: 1.2087
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement cfa110 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-313.3997891616479;dx=-0.1448783947526882
New Minimum: -313.3997891616479 > -485.11439224558245
END: th(7200.000000000001)=-485.11439224558245; dx=-0.019176138415352567 evalInputDelta=171.71460308393455
Fitness changed from -313.3997891616479 to -485.11439224558245
Iteration 3 complete. Error: -485.11439224558245 Total: 3.1602; Orientation: 0.0162; Line Search: 2.5903
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 4b344805 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.604e+03, gradient 3.243e-01, dot -0.165; [8e833f84-e1a4-4fb6-9368-8e7e0b1a0b7c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -485.11439224558245, -313.3997891616479, -171.37744074497917, -93.3044883114553
LBFGS Accumulation History: 3 points
Removed measurement 4b344805 to history. Total: 3
th(0)=-485.11439224558245;dx=-0.10504097414582791
New Minimum: -485.11439224558245 > -647.7352103934282
END: th(15511.929768229566)=-647.7352103934282; dx=-0.00914547933671182 evalInputDelta=162.6208181478458
Fitness changed from -485.11439224558245 to -647.7352103934282
Iteration 4 complete. Error: -647.7352103934282 Total: 2.0866; Orientation: 0.2987; Line Search: 1.2178
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 427d9f43 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.981e+03, gradient 3.004e-01, dot -0.125; [8e833f84-e1a4-4fb6-9368-8e7e0b1a0b7c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -647.7352103934282, -313.3997891616479, -171.37744074497917, -93.3044883114553
LBFGS Accumulation History: 3 points
Removed measurement 427d9f43 to history. Total: 3
th(0)=-647.7352103934282;dx=-0.09009618157594732
New Minimum: -647.7352103934282 > -807.7897318545997
END: th(33419.43960201202)=-807.7897318545997; dx=-0.002405416497319511 evalInputDelta=160.05452146117148
Fitness changed from -647.7352103934282 to -807.7897318545997
Iteration 5 complete. Error: -807.7897318545997 Total: 2.0844; Orientation: 0.2926; Line Search: 1.2196
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 5c19d1fa to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.692e+03, gradient 2.730e-01, dot -0.089; [8e833f84-e1a4-4fb6-9368-8e7e0b1a0b7c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -807.7897318545997, -313.3997891616479, -171.37744074497917, -93.3044883114553
LBFGS Accumulation History: 3 points
Removed measurement 5c19d1fa to history. Total: 3
th(0)=-807.7897318545997;dx=-0.07440356255763911
Armijo: th(72000.00000000003)=-706.9537792724449; dx=2.021746139801631E-5 evalInputDelta=-100.83595258215485
New Minimum: -807.7897318545997 > -1218.4270924023654
END: th(36000.000000000015)=-1218.4270924023654; dx=-0.006808011740283549 evalInputDelta=410.63736054776564
Fitness changed from -807.7897318545997 to -1218.4270924023654
Iteration 6 complete. Error: -1218.4270924023654 Total: 2.6550; Orientation: 0.2917; Line Search: 1.7495
Adding measurement 40da0be9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.106e+04, gradient 2.654e-01, dot -0.106; [8e833f84-e1a4-4fb6-9368-8e7e0b1a0b7c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1218.4270924023654, -313.3997891616479, -171.37744074497917, -93.3044883114553
LBFGS Accumulation History: 3 points
Removed measurement 40da0be9 to history. Total: 3
th(0)=-1218.4270924023654;dx=-0.07035710774139779
Armijo: th(77559.64884114785)=-1046.7814999488464; dx=0.0012945494215595831 evalInputDelta=-171.64559245351893
New Minimum: -1218.4270924023654 > -1726.276836291098
END: th(38779.824420573925)=-1726.276836291098; dx=-0.008003269722741076 evalInputDelta=507.84974388873275
Fitness changed from -1218.4270924023654 to -1726.276836291098
Iteration 7 complete. Error: -1726.276836291098 Total: 2.6763; Orientation: 0.3021; Line Search: 1.8316
Adding measurement 7e1e7479 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.330e+04, gradient 2.703e-01, dot -0.124; [8e833f84-e1a4-4fb6-9368-8e7e0b1a0b7c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1726.276836291098, -313.3997891616479, -171.37744074497917, -93.3044883114553
LBFGS Accumulation History: 3 points
Removed measurement 7e1e7479 to history. Total: 3
th(0)=-1726.276836291098;dx=-0.0729542407924038
Armijo: th(83548.59900503006)=-1336.8783860209971; dx=0.002353586963419217 evalInputDelta=-389.39845027010097
New Minimum: -1726.276836291098 > -2165.6419279354786
END: th(41774.29950251503)=-2165.6419279354786; dx=-0.007604307175908271 evalInputDelta=439.3650916443805
Fitness changed from -1726.276836291098 to -2165.6419279354786
Iteration 8 complete. Error: -2165.6419279354786 Total: 2.9233; Orientation: 0.2837; Line Search: 1.8063
Adding measurement 7536ded to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.140e+04, gradient 2.744e-01, dot -0.138; [8e833f84-e1a4-4fb6-9368-8e7e0b1a0b7c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2165.6419279354786, -313.3997891616479, -171.37744074497917, -93.3044883114553
LBFGS Accumulation History: 3 points
Removed measurement 7536ded to history. Total: 3
th(0)=-2165.6419279354786;dx=-0.07529792566659747
Armijo: th(90000.00000000004)=-1502.3157392969122; dx=0.0038006585892663394 evalInputDelta=-663.3261886385665
New Minimum: -2165.6419279354786 > -2525.146059996571
END: th(45000.00000000002)=-2525.146059996571; dx=-0.005655705990536476 evalInputDelta=359.50413206109215
Fitness changed from -2165.6419279354786 to -2525.146059996571
Iteration 9 complete. Error: -2525.146059996571 Total: 2.6616; Orientation: 0.2933; Line Search: 1.8122
Adding measurement 516a0895 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.542e+05, gradient 2.814e-01, dot -0.148; [8e833f84-e1a4-4fb6-9368-8e7e0b1a0b7c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2525.146059996571, -313.3997891616479, -171.37744074497917, -93.3044883114553
LBFGS Accumulation History: 3 points
Removed measurement 516a0895 to history. Total: 3
th(0)=-2525.146059996571;dx=-0.07908481369168435
Armijo: th(96949.56105143482)=-1559.795288456639; dx=0.0049105984720588065 evalInputDelta=-965.3507715399319
New Minimum: -2525.146059996571 > -2796.101134683063
END: th(48474.78052571741)=-2796.101134683063; dx=-0.0032858557806043553 evalInputDelta=270.955074686492
Fitness changed from -2525.146059996571 to -2796.101134683063
Iteration 10 complete. Error: -2796.101134683063 Total: 2.6903; Orientation: 0.2930; Line Search: 1.8208
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 7cda25d3 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.022e+05, gradient 2.920e-01, dot 0.158; [8e833f84-e1a4-4fb6-9368-8e7e0b1a0b7c = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-2796.101134683063;dx=-4729.785107746345
Armijo: th(104435.74875628759)=-537.8027854851055; dx=0.008167339520752818 evalInputDelta=-2258.2983491979576
Armijo: th(52217.874378143795)=-538.0757741973562; dx=0.016372555763109708 evalInputDelta=-2258.0253604857066
Armijo: th(17405.95812604793)=-538.7590316911078; dx=0.049077210515779554 evalInputDelta=-2257.342102991955
Armijo: th(4351.489531511983)=-540.5454764425765; dx=0.19482488894010708 evalInputDelta=-2255.555658240486
Armijo: th(870.2979063023965)=-546.5981815147438; dx=0.970042263193344 evalInputDelta=-2249.502953168319
Armijo: th(145.04965105039943)=-563.7755507257816; dx=5.700960098341877 evalInputDelta=-2232.325583957281
Armijo: th(20.72137872148563)=-619.5872091490605; dx=36.952261359980994 evalInputDelta=-2176.5139255340023
Armijo: th(2.590172340185704)=-925.8653081740463; dx=208.72019284471907 evalInputDelta=-1870.2358265090165
Armijo: th(0.28779692668730045)=-2154.3250891806997; dx=675.3783600141118 evalInputDelta=-641.7760455023631
Armijo: th(0.028779692668730046)=-2779.5644582371265; dx=-603.1297437281099 evalInputDelta=-16.536676445936337
New Minimum: -2796.101134683063 > -2796.2211459776076
END: th(0.002616335697157277)=-2796.2211459776076; dx=-1178.330521039527 evalInputDelta=0.12001129454483817
Fitness changed from -2796.101134683063 to -2796.2211459776076
Iteration 11 complete. Error: -2796.2211459776076 Total: 8.6222; Orientation: 0.3034; Line Search: 7.7577
Adding measurement 853a897 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 4.619e+06, gradient 2.913e-01, dot -0.144; [8e833f84-e1a4-4fb6-9368-8e7e0b1a0b7c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2796.2211459776076, -2796.101134683063, -313.3997891616479, -171.37744074497917, -93.3044883114553
Rejected: LBFGS Orientation magnitude: 1.504e+07, gradient 2.913e-01, dot -0.161; [8e833f84-e1a4-4fb6-9368-8e7e0b1a0b7c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2796.2211459776076, -2796.101134683063, -313.3997891616479, -171.37744074497917
LBFGS Accumulation History: 3 points
Removed measurement 853a897 to history. Total: 4
Removed measurement 7cda25d3 to history. Total: 3
th(0)=-2796.2211459776076;dx=-0.08480543249746662
New Minimum: -2796.2211459776076 > -2796.221318231822
END: th(0.00563672438672439)=-2796.221318231822; dx=-0.06735800242385968 evalInputDelta=1.722542142488237E-4
Fitness changed from -2796.2211459776076 to -2796.221318231822
Iteration 12 complete. Error: -2796.221318231822 Total: 2.4073; Orientation: 0.6394; Line Search: 1.2105
Adding measurement 20e2520a to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.038e+05, gradient 2.913e-01, dot 0.158; [8e833f84-e1a4-4fb6-9368-8e7e0b1a0b7c = 1.000/1.000e+00]
th(0)=-2796.221318231822;dx=-4770.859127055119
Armijo: th(0.012143954556907723)=-2792.1633304765514; dx=-912.8604760236515 evalInputDelta=-4.05798775527046
Armijo: th(0.006071977278453862)=-2795.1388016183014; dx=-1062.704092472612 evalInputDelta=-1.0825166135205109
Armijo: th(0.002023992426151287)=-2796.073780786192; dx=-1149.326767528641 evalInputDelta=-0.14753744562995053
Armijo: th(5.059981065378218E-4)=-2796.203865297668; dx=-1187.9691030438119 evalInputDelta=-0.01745293415388005
Armijo: th(1.0119962130756435E-4)=-2796.218799373862; dx=-1215.4437668875573 evalInputDelta=-0.0025188579597852367
Armijo: th(1.6866603551260725E-5)=-2796.2209527076925; dx=-1209.8583940462524 evalInputDelta=-3.655241293927247E-4
Armijo: th(2.4095147930372463E-6)=-2796.2212359579435; dx=-1207.1936301706387 evalInputDelta=-8.227387843362521E-5
New Minimum: -2796.221318231822 > -2796.2213378981355
END: th(3.011893491296558E-7)=-2796.2213378981355; dx=-1156.0176104518855 evalInputDelta=1.966631361938198E-5
Fitness changed from -2796.221318231822 to -2796.2213378981355
Iteration 13 complete. Error: -2796.2213378981355 Total: 6.5998; Orientation: 0.2989; Line Search: 5.7266
Adding measurement 266f1e59 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 4.633e+11, gradient 2.913e-01, dot -0.144; [8e833f84-e1a4-4fb6-9368-8e7e0b1a0b7c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2796.2213378981355, -2796.221318231822, -313.3997891616479, -171.37744074497917, -93.3044883114553
Rejected: LBFGS Orientation magnitude: 1.875e+12, gradient 2.913e-01, dot -0.160; [8e833f84-e1a4-4fb6-9368-8e7e0b1a0b7c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2796.2213378981355, -2796.221318231822, -313.3997891616479, -171.37744074497917
LBFGS Accumulation History: 3 points
Removed measurement 266f1e59 to history. Total: 4
Removed measurement 20e2520a to history. Total: 3
th(0)=-2796.2213378981355;dx=-0.08479846860421043
Armijo: th(6.488927820330548E-7)=-2796.2211656439313; dx=-0.06748301874954346 evalInputDelta=-1.7225420424438198E-4
Armijo: th(3.244463910165274E-7)=-2796.2211656439313; dx=-0.06746612075646609 evalInputDelta=-1.7225420424438198E-4
END: th(1.0814879700550914E-7)=-2796.2213378981355; dx=-0.06748192190638577 evalInputDelta=0.0
Fitness changed from -2796.2213378981355 to -2796.2213378981355
Static Iteration Total: 3.6048; Orientation: 0.6331; Line Search: 2.4008
Iteration 14 failed. Error: -2796.2213378981355
Previous Error: 0.0 -> -2796.2213378981355
Retrying iteration 14
Reset training subject: 219254415277500
Adding measurement 5c179062 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-2796.2213378981355;dx=-0.08479489909279875
Armijo: th(2.329995199538852E-7)=-2796.2211656439313; dx=-0.06748003734644145 evalInputDelta=-1.7225420424438198E-4
END: th(1.164997599769426E-7)=-2796.2213378981355; dx=-0.06745621944767616 evalInputDelta=0.0
Fitness changed from -2796.2213378981355 to -2796.2213378981355
Static Iteration Total: 3.3062; Orientation: 0.0162; Line Search: 2.1532
Iteration 15 failed. Error: -2796.2213378981355
Previous Error: 0.0 -> -2796.2213378981355
Optimization terminated 15
Final threshold in iteration 15: -2796.2213378981355 (> -Infinity) after 53.306s (< 5400.000s)
Returns
-2796.2213378981355