BasicOptimizer.scala:89 executed in 49.07 seconds (0.992 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 214930382345700
Reset training subject: 214931009331200
Adding measurement 5fe0fe09 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-8.024326612121163;dx=-6.886937961094232E-4
New Minimum: -8.024326612121163 > -8.025274164016007
WOLFE (weak): th(2.154434690031884)=-8.025274164016007; dx=-6.890068980371844E-4 evalInputDelta=9.475518948445227E-4
New Minimum: -8.025274164016007 > -8.026221284492589
WOLFE (weak): th(4.308869380063768)=-8.026221284492589; dx=-6.893675317171361E-4 evalInputDelta=0.0018946723714261537
New Minimum: -8.026221284492589 > -8.030019385239335
WOLFE (weak): th(12.926608140191302)=-8.030019385239335; dx=-6.875291896314996E-4 evalInputDelta=0.0056927731181719565
New Minimum: -8.030019385239335 > -8.047129479231284
WOLFE (weak): th(51.70643256076521)=-8.047129479231284; dx=-6.877403538111798E-4 evalInputDelta=0.022802867110121028
New Minimum: -8.047129479231284 > -8.13922642594457
WOLFE (weak): th(258.53216280382605)=-8.13922642594457; dx=-6.844974294525824E-4 evalInputDelta=0.1148998138234063
New Minimum: -8.13922642594457 > -8.73642211664129
WOLFE (weak): th(1551.1929768229563)=-8.73642211664129; dx=-6.627624960050993E-4 evalInputDelta=0.7120955045201267
New Minimum: -8.73642211664129 > -11.971334162997085
END: th(10858.350837760694)=-11.971334162997085; dx=-4.050549966868472E-4 evalInputDelta=3.947007550875922
Fitness changed from -8.024326612121163 to -11.971334162997085
Iteration 1 complete. Error: -11.971334162997085 Total: 6.8514; Orientation: 0.0158; Line Search: 4.8787
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 32715974 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-11.971334162997085;dx=-9.903943248948106E-4
New Minimum: -11.971334162997085 > -18.943189089572094
END: th(23393.607721408407)=-18.943189089572094; dx=-3.008384559207482E-4 evalInputDelta=6.971854926575009
Fitness changed from -11.971334162997085 to -18.943189089572094
Iteration 2 complete. Error: -18.943189089572094 Total: 1.8404; Orientation: 0.0160; Line Search: 1.2297
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 608359b6 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-18.943189089572094;dx=-8.609593648359415E-4
New Minimum: -18.943189089572094 > -27.69040758691285
END: th(50400.00000000001)=-27.69040758691285; dx=-1.854979206947622E-4 evalInputDelta=8.747218497340757
Fitness changed from -18.943189089572094 to -27.69040758691285
Iteration 3 complete. Error: -27.69040758691285 Total: 1.7915; Orientation: 0.0159; Line Search: 1.2176
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 4d16de1b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.045e+03, gradient 2.936e-02, dot -0.205; [72ae488b-6b67-442d-9a38-00299688dc27 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -27.69040758691285, -18.943189089572094, -11.971334162997085, -8.024326612121163
LBFGS Accumulation History: 3 points
Removed measurement 4d16de1b to history. Total: 3
th(0)=-27.69040758691285;dx=-8.617413081716656E-4
New Minimum: -27.69040758691285 > -31.563589948704063
END: th(108583.50837760697)=-31.563589948704063; dx=-5.1019630269782397E-5 evalInputDelta=3.873182361791212
Fitness changed from -27.69040758691285 to -31.563589948704063
Iteration 4 complete. Error: -31.563589948704063 Total: 2.1129; Orientation: 0.3027; Line Search: 1.2317
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 4f852ab3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.249e+03, gradient 2.997e-02, dot -0.136; [72ae488b-6b67-442d-9a38-00299688dc27 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -31.563589948704063, -18.943189089572094, -11.971334162997085, -8.024326612121163
LBFGS Accumulation History: 3 points
Removed measurement 4f852ab3 to history. Total: 3
th(0)=-31.563589948704063;dx=-8.983137617716975E-4
Armijo: th(233936.07721408413)=-24.354353617469293; dx=1.045073282140975E-5 evalInputDelta=-7.20923633123477
New Minimum: -31.563589948704063 > -40.79151272580282
END: th(116968.03860704206)=-40.79151272580282; dx=-6.468968846490245E-5 evalInputDelta=9.227922777098755
Fitness changed from -31.563589948704063 to -40.79151272580282
Iteration 5 complete. Error: -40.79151272580282 Total: 3.0675; Orientation: 0.2962; Line Search: 1.8520
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 611fce1c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.664e+03, gradient 3.013e-02, dot -0.120; [72ae488b-6b67-442d-9a38-00299688dc27 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -40.79151272580282, -18.943189089572094, -11.971334162997085, -8.024326612121163
LBFGS Accumulation History: 3 points
Removed measurement 611fce1c to history. Total: 3
th(0)=-40.79151272580282;dx=-9.078314118493167E-4
Armijo: th(252000.0000000001)=-31.199637273803212; dx=3.1369257472104825E-5 evalInputDelta=-9.591875451999606
New Minimum: -40.79151272580282 > -53.662882166570796
END: th(126000.00000000004)=-53.662882166570796; dx=-7.547280621037777E-5 evalInputDelta=12.871369440767978
Fitness changed from -40.79151272580282 to -53.662882166570796
Iteration 6 complete. Error: -53.662882166570796 Total: 2.7333; Orientation: 0.2946; Line Search: 1.8519
Adding measurement 701156e1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.439e+03, gradient 3.001e-02, dot -0.120; [72ae488b-6b67-442d-9a38-00299688dc27 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -53.662882166570796, -18.943189089572094, -11.971334162997085, -8.024326612121163
LBFGS Accumulation History: 3 points
Removed measurement 701156e1 to history. Total: 3
th(0)=-53.662882166570796;dx=-9.00619522495423E-4
Armijo: th(271458.77094401745)=-38.95284250206631; dx=2.7944685970295332E-5 evalInputDelta=-14.710039664504485
New Minimum: -53.662882166570796 > -64.71475589551748
END: th(135729.38547200873)=-64.71475589551748; dx=-8.077504094723701E-5 evalInputDelta=11.051873728946688
Fitness changed from -53.662882166570796 to -64.71475589551748
Iteration 7 complete. Error: -64.71475589551748 Total: 2.6723; Orientation: 0.2891; Line Search: 1.8300
Adding measurement 9624d9e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.784e+03, gradient 2.883e-02, dot -0.129; [72ae488b-6b67-442d-9a38-00299688dc27 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -64.71475589551748, -18.943189089572094, -11.971334162997085, -8.024326612121163
LBFGS Accumulation History: 3 points
Removed measurement 9624d9e to history. Total: 3
th(0)=-64.71475589551748;dx=-8.313910976296733E-4
Armijo: th(292420.0965176052)=-50.59874446140782; dx=3.315785898953058E-5 evalInputDelta=-14.116011434109666
New Minimum: -64.71475589551748 > -78.26345196092609
END: th(146210.0482588026)=-78.26345196092609; dx=-1.0197851293676055E-4 evalInputDelta=13.548696065408606
Fitness changed from -64.71475589551748 to -78.26345196092609
Iteration 8 complete. Error: -78.26345196092609 Total: 2.6471; Orientation: 0.2894; Line Search: 1.7734
Adding measurement 695b18d2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.669e+04, gradient 2.812e-02, dot -0.147; [72ae488b-6b67-442d-9a38-00299688dc27 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -78.26345196092609, -18.943189089572094, -11.971334162997085, -8.024326612121163
LBFGS Accumulation History: 3 points
Removed measurement 695b18d2 to history. Total: 3
th(0)=-78.26345196092609;dx=-7.905544552917578E-4
Armijo: th(315000.0000000001)=-62.44625754234565; dx=3.388357424324989E-5 evalInputDelta=-15.817194418580442
New Minimum: -78.26345196092609 > -92.96688501881502
END: th(157500.00000000006)=-92.96688501881502; dx=-1.0390095540838106E-4 evalInputDelta=14.70343305788893
Fitness changed from -78.26345196092609 to -92.96688501881502
Iteration 9 complete. Error: -92.96688501881502 Total: 2.8577; Orientation: 0.2763; Line Search: 1.7720
Adding measurement 26bcdfad to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.811e+04, gradient 2.823e-02, dot -0.154; [72ae488b-6b67-442d-9a38-00299688dc27 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -92.96688501881502, -18.943189089572094, -11.971334162997085, -8.024326612121163
LBFGS Accumulation History: 3 points
Removed measurement 26bcdfad to history. Total: 3
th(0)=-92.96688501881502;dx=-7.967062478275505E-4
Armijo: th(339323.4636800218)=-71.1491425094137; dx=3.311718046937106E-5 evalInputDelta=-21.817742509401313
New Minimum: -92.96688501881502 > -105.80860273342637
END: th(169661.7318400109)=-105.80860273342637; dx=-1.0084739108749249E-4 evalInputDelta=12.841717714611349
Fitness changed from -92.96688501881502 to -105.80860273342637
Iteration 10 complete. Error: -105.80860273342637 Total: 2.7011; Orientation: 0.2887; Line Search: 1.8328
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 160d17f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.119e+04, gradient 2.835e-02, dot -0.170; [72ae488b-6b67-442d-9a38-00299688dc27 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -105.80860273342637, -18.943189089572094, -11.971334162997085, -8.024326612121163
LBFGS Accumulation History: 3 points
Removed measurement 160d17f to history. Total: 3
th(0)=-105.80860273342637;dx=-8.037411440383791E-4
Armijo: th(365525.1206470065)=-72.26147715677901; dx=4.427260128160548E-5 evalInputDelta=-33.54712557664736
New Minimum: -105.80860273342637 > -116.20055654316329
END: th(182762.56032350325)=-116.20055654316329; dx=-8.803254537241276E-5 evalInputDelta=10.391953809736918
Fitness changed from -105.80860273342637 to -116.20055654316329
Iteration 11 complete. Error: -116.20055654316329 Total: 3.0174; Orientation: 0.5730; Line Search: 1.8652
Adding measurement 2c24e37e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.784e+05, gradient 2.926e-02, dot -0.159; [72ae488b-6b67-442d-9a38-00299688dc27 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -116.20055654316329, -18.943189089572094, -11.971334162997085, -8.024326612121163
LBFGS Accumulation History: 3 points
Removed measurement 2c24e37e to history. Total: 3
th(0)=-116.20055654316329;dx=-8.560523465357532E-4
Armijo: th(393750.0000000002)=-72.95839881182856; dx=6.486818415479958E-5 evalInputDelta=-43.24215773133473
New Minimum: -116.20055654316329 > -125.7755744951485
END: th(196875.0000000001)=-125.7755744951485; dx=-7.878223413480333E-5 evalInputDelta=9.575017951985217
Fitness changed from -116.20055654316329 to -125.7755744951485
Iteration 12 complete. Error: -125.7755744951485 Total: 2.6801; Orientation: 0.2855; Line Search: 1.8501
Adding measurement 38341dcb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.225e+06, gradient 2.946e-02, dot -0.155; [72ae488b-6b67-442d-9a38-00299688dc27 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -125.7755744951485, -18.943189089572094, -11.971334162997085, -8.024326612121163
LBFGS Accumulation History: 3 points
Removed measurement 38341dcb to history. Total: 3
th(0)=-125.7755744951485;dx=-8.676321219090132E-4
Armijo: th(424154.3296000273)=-73.06769343502107; dx=7.294385915706485E-5 evalInputDelta=-52.707881060127434
New Minimum: -125.7755744951485 > -132.37497290262564
END: th(212077.16480001365)=-132.37497290262564; dx=-6.268143741464115E-5 evalInputDelta=6.599398407477139
Fitness changed from -125.7755744951485 to -132.37497290262564
Iteration 13 complete. Error: -132.37497290262564 Total: 3.0228; Orientation: 0.2970; Line Search: 1.9161
Adding measurement ec77fac to history. Total: 3
Accepted: LBFGS Orientation magnitude: 6.323e+04, gradient 2.918e-02, dot 0.169; [72ae488b-6b67-442d-9a38-00299688dc27 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-132.37497290262564;dx=-311.56004113238765
Armijo: th(456906.40080875816)=666.5245492466609; dx=7.564745542510969E-4 evalInputDelta=-798.8995221492864
Armijo: th(228453.20040437908)=666.5185512746748; dx=0.0015124994807797416 evalInputDelta=-798.8935241773004
Armijo: th(76151.06680145969)=666.5063623120093; dx=0.004539012714580279 evalInputDelta=-798.8813352146349
Armijo: th(19037.766700364922)=666.3085341016642; dx=0.01814861211199322 evalInputDelta=-798.6835070042898
Armijo: th(3807.5533400729846)=665.2327090512548; dx=0.09061282316992995 evalInputDelta=-797.6076819538805
Armijo: th(634.5922233454975)=658.5239281592018; dx=0.5377306431728407 evalInputDelta=-790.8989010618275
Armijo: th(90.65603190649963)=621.8486442349699; dx=3.5823763965273816 evalInputDelta=-754.2236171375955
Armijo: th(11.332003988312454)=487.55174599882884; dx=23.72713662946889 evalInputDelta=-619.9267189014545
Armijo: th(1.2591115542569393)=138.2481383927126; dx=105.4582679351973 evalInputDelta=-270.6231112953383
Armijo: th(0.12591115542569392)=-116.5095642229243; dx=-7.074972180545885 evalInputDelta=-15.865408679701346
New Minimum: -132.37497290262564 > -132.4311906219651
END: th(0.011446468675063083)=-132.4311906219651; dx=-163.22353963482078 evalInputDelta=0.05621771933945752
Fitness changed from -132.37497290262564 to -132.4311906219651
Iteration 14 complete. Error: -132.4311906219651 Total: 8.3438; Orientation: 0.3189; Line Search: 7.4130
Adding measurement 4003db85 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 2.730e+05, gradient 2.912e-02, dot -0.141; [72ae488b-6b67-442d-9a38-00299688dc27 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -132.4311906219651, -132.37497290262564, -18.943189089572094, -11.971334162997085, -8.024326612121163
Rejected: LBFGS Orientation magnitude: 6.690e+05, gradient 2.912e-02, dot -0.151; [72ae488b-6b67-442d-9a38-00299688dc27 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -132.4311906219651, -132.37497290262564, -18.943189089572094, -11.971334162997085
LBFGS Accumulation History: 3 points
Removed measurement 4003db85 to history. Total: 4
Removed measurement ec77fac to history. Total: 3
th(0)=-132.4311906219651;dx=-8.480255753180637E-4
New Minimum: -132.4311906219651 > -132.43121030536307
END: th(0.024660669191919202)=-132.43121030536307; dx=-7.516431722377379E-4 evalInputDelta=1.9683397965764016E-5
Fitness changed from -132.4311906219651 to -132.43121030536307
Iteration 15 complete. Error: -132.43121030536307 Total: 2.7095; Orientation: 0.6197; Line Search: 1.1869
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -132.43121030536307 (> -Infinity) after 49.071s (< 5400.000s)
Returns
-132.43121030536307