BasicOptimizer.scala:89 executed in 47.39 seconds (1.158 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 216712816758000
Reset training subject: 216713421937200
Adding measurement 453b96a3 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-193.82148882734856;dx=-0.07136465172692052
New Minimum: -193.82148882734856 > -193.954307400737
WOLFE (weak): th(2.154434690031884)=-193.954307400737; dx=-0.07149888339336744 evalInputDelta=0.13281857338844816
New Minimum: -193.954307400737 > -194.0872460059571
WOLFE (weak): th(4.308869380063768)=-194.0872460059571; dx=-0.07103729423927566 evalInputDelta=0.2657571786085384
New Minimum: -194.0872460059571 > -194.61874065408688
WOLFE (weak): th(12.926608140191302)=-194.61874065408688; dx=-0.07140049901514614 evalInputDelta=0.797251826738318
New Minimum: -194.61874065408688 > -196.99719906121945
WOLFE (weak): th(51.70643256076521)=-196.99719906121945; dx=-0.07072424579173872 evalInputDelta=3.1757102338708876
New Minimum: -196.99719906121945 > -209.1869451342484
WOLFE (weak): th(258.53216280382605)=-209.1869451342484; dx=-0.06763885692287062 evalInputDelta=15.36545630689983
New Minimum: -209.1869451342484 > -269.6965385945564
END: th(1551.1929768229563)=-269.6965385945564; dx=-0.04979755390362167 evalInputDelta=75.87504976720783
Fitness changed from -193.82148882734856 to -269.6965385945564
Iteration 1 complete. Error: -269.6965385945564 Total: 5.9661; Orientation: 0.0157; Line Search: 4.2353
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 3e46203f to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-269.6965385945564;dx=-0.08122704459104262
New Minimum: -269.6965385945564 > -378.960802479516
END: th(3341.943960201201)=-378.960802479516; dx=-0.030416435861680274 evalInputDelta=109.26426388495963
Fitness changed from -269.6965385945564 to -378.960802479516
Iteration 2 complete. Error: -378.960802479516 Total: 1.8208; Orientation: 0.0163; Line Search: 1.2210
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 363beaf5 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-378.960802479516;dx=-0.08992111394463652
New Minimum: -378.960802479516 > -449.42436295178516
END: th(7200.000000000001)=-449.42436295178516; dx=-0.011767532452078252 evalInputDelta=70.46356047226914
Fitness changed from -378.960802479516 to -449.42436295178516
Iteration 3 complete. Error: -449.42436295178516 Total: 1.8063; Orientation: 0.0162; Line Search: 1.2304
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 711fd0b2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.166e+03, gradient 3.119e-01, dot -0.214; [950ae0c5-0b4c-4368-8d49-fef1bb6f05c7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -449.42436295178516, -378.960802479516, -269.6965385945564, -193.82148882734856
LBFGS Accumulation History: 3 points
Removed measurement 711fd0b2 to history. Total: 3
th(0)=-449.42436295178516;dx=-0.09685600356705473
Armijo: th(15511.929768229566)=-407.60854606088776; dx=-1.4027092687228705E-4 evalInputDelta=-41.815816890897395
New Minimum: -449.42436295178516 > -554.8121516335092
END: th(7755.964884114783)=-554.8121516335092; dx=-0.009426533099051932 evalInputDelta=105.38778868172403
Fitness changed from -449.42436295178516 to -554.8121516335092
Iteration 4 complete. Error: -554.8121516335092 Total: 2.9369; Orientation: 0.2950; Line Search: 1.8486
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 19061f05 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.387e+03, gradient 3.064e-01, dot -0.179; [950ae0c5-0b4c-4368-8d49-fef1bb6f05c7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -554.8121516335092, -378.960802479516, -269.6965385945564, -193.82148882734856
LBFGS Accumulation History: 3 points
Removed measurement 19061f05 to history. Total: 3
th(0)=-554.8121516335092;dx=-0.09374782223273037
Armijo: th(16709.71980100601)=-506.38914483533426; dx=0.001861874741867075 evalInputDelta=-48.423006798174924
New Minimum: -554.8121516335092 > -701.5652577416353
END: th(8354.859900503005)=-701.5652577416353; dx=-0.010042789538923073 evalInputDelta=146.75310610812608
Fitness changed from -554.8121516335092 to -701.5652577416353
Iteration 5 complete. Error: -701.5652577416353 Total: 2.6517; Orientation: 0.3001; Line Search: 1.7913
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 13072652 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.498e+03, gradient 2.956e-01, dot -0.177; [950ae0c5-0b4c-4368-8d49-fef1bb6f05c7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -701.5652577416353, -378.960802479516, -269.6965385945564, -193.82148882734856
LBFGS Accumulation History: 3 points
Removed measurement 13072652 to history. Total: 3
th(0)=-701.5652577416353;dx=-0.08714764844872111
Armijo: th(18000.000000000007)=-619.9308020985046; dx=0.004012007953164288 evalInputDelta=-81.63445564313065
New Minimum: -701.5652577416353 > -849.7559586097998
END: th(9000.000000000004)=-849.7559586097998; dx=-0.009973610333470523 evalInputDelta=148.19070086816453
Fitness changed from -701.5652577416353 to -849.7559586097998
Iteration 6 complete. Error: -849.7559586097998 Total: 2.7321; Orientation: 0.2974; Line Search: 1.8523
Adding measurement 4302c27 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.928e+03, gradient 2.878e-01, dot -0.185; [950ae0c5-0b4c-4368-8d49-fef1bb6f05c7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -849.7559586097998, -378.960802479516, -269.6965385945564, -193.82148882734856
LBFGS Accumulation History: 3 points
Removed measurement 4302c27 to history. Total: 3
th(0)=-849.7559586097998;dx=-0.08266813636810105
Armijo: th(19389.912210286962)=-752.6017589235056; dx=0.0036168682938607735 evalInputDelta=-97.15419968629419
New Minimum: -849.7559586097998 > -1000.5497090623351
END: th(9694.956105143481)=-1000.5497090623351; dx=-0.00954265683272169 evalInputDelta=150.79375045253528
Fitness changed from -849.7559586097998 to -1000.5497090623351
Iteration 7 complete. Error: -1000.5497090623351 Total: 2.6603; Orientation: 0.2956; Line Search: 1.7896
Adding measurement 52e76110 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.325e+04, gradient 2.762e-01, dot -0.194; [950ae0c5-0b4c-4368-8d49-fef1bb6f05c7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1000.5497090623351, -378.960802479516, -269.6965385945564, -193.82148882734856
LBFGS Accumulation History: 3 points
Removed measurement 52e76110 to history. Total: 3
th(0)=-1000.5497090623351;dx=-0.07614866371807998
Armijo: th(20887.149751257515)=-869.4496099371212; dx=0.004411712165136157 evalInputDelta=-131.1000991252139
New Minimum: -1000.5497090623351 > -1145.8382919595608
END: th(10443.574875628758)=-1145.8382919595608; dx=-0.0098356440012341 evalInputDelta=145.28858289722575
Fitness changed from -1000.5497090623351 to -1145.8382919595608
Iteration 8 complete. Error: -1145.8382919595608 Total: 3.2750; Orientation: 0.8189; Line Search: 1.8746
Adding measurement 2e39a5e4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.824e+04, gradient 2.793e-01, dot -0.191; [950ae0c5-0b4c-4368-8d49-fef1bb6f05c7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1145.8382919595608, -378.960802479516, -269.6965385945564, -193.82148882734856
LBFGS Accumulation History: 3 points
Removed measurement 2e39a5e4 to history. Total: 3
th(0)=-1145.8382919595608;dx=-0.07784391665770513
Armijo: th(22500.00000000001)=-997.6542498936855; dx=0.004745415830606353 evalInputDelta=-148.18404206587536
New Minimum: -1145.8382919595608 > -1282.0802393883887
END: th(11250.000000000005)=-1282.0802393883887; dx=-0.008189795189262924 evalInputDelta=136.24194742882787
Fitness changed from -1145.8382919595608 to -1282.0802393883887
Iteration 9 complete. Error: -1282.0802393883887 Total: 2.6490; Orientation: 0.2847; Line Search: 1.8272
Adding measurement 10e724d3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.761e+04, gradient 2.742e-01, dot -0.200; [950ae0c5-0b4c-4368-8d49-fef1bb6f05c7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1282.0802393883887, -378.960802479516, -269.6965385945564, -193.82148882734856
LBFGS Accumulation History: 3 points
Removed measurement 10e724d3 to history. Total: 3
th(0)=-1282.0802393883887;dx=-0.07496752387697586
Armijo: th(24237.390262858706)=-1092.6152530167901; dx=0.0049063950015926176 evalInputDelta=-189.46498637159857
New Minimum: -1282.0802393883887 > -1405.4755811218706
END: th(12118.695131429353)=-1405.4755811218706; dx=-0.007450009477609005 evalInputDelta=123.39534173348193
Fitness changed from -1282.0802393883887 to -1405.4755811218706
Iteration 10 complete. Error: -1405.4755811218706 Total: 2.7479; Orientation: 0.3089; Line Search: 1.8829
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 42378510 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.579e+04, gradient 2.798e-01, dot -0.196; [950ae0c5-0b4c-4368-8d49-fef1bb6f05c7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1405.4755811218706, -378.960802479516, -269.6965385945564, -193.82148882734856
LBFGS Accumulation History: 3 points
Removed measurement 42378510 to history. Total: 3
th(0)=-1405.4755811218706;dx=-0.07815280335122474
Armijo: th(26108.937189071898)=-1159.7819184662026; dx=0.0057137453188928515 evalInputDelta=-245.69366265566805
New Minimum: -1405.4755811218706 > -1511.4695320507435
END: th(13054.468594535949)=-1511.4695320507435; dx=-0.005843206038178909 evalInputDelta=105.99395092887289
Fitness changed from -1405.4755811218706 to -1511.4695320507435
Iteration 11 complete. Error: -1511.4695320507435 Total: 2.6585; Orientation: 0.2919; Line Search: 1.7812
Adding measurement 31ef3513 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.886e+04, gradient 2.752e-01, dot -0.207; [950ae0c5-0b4c-4368-8d49-fef1bb6f05c7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1511.4695320507435, -378.960802479516, -269.6965385945564, -193.82148882734856
LBFGS Accumulation History: 3 points
Removed measurement 31ef3513 to history. Total: 3
th(0)=-1511.4695320507435;dx=-0.07547817335530338
Armijo: th(28125.00000000002)=-1265.4859088548608; dx=0.007006508315281734 evalInputDelta=-245.98362319588273
New Minimum: -1511.4695320507435 > -1648.7141133465275
END: th(14062.50000000001)=-1648.7141133465275; dx=-0.0064196753961737775 evalInputDelta=137.24458129578397
Fitness changed from -1511.4695320507435 to -1648.7141133465275
Iteration 12 complete. Error: -1648.7141133465275 Total: 2.9061; Orientation: 0.2752; Line Search: 1.8427
Adding measurement 3e76bd23 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.160e+05, gradient 2.694e-01, dot -0.210; [950ae0c5-0b4c-4368-8d49-fef1bb6f05c7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1648.7141133465275, -378.960802479516, -269.6965385945564, -193.82148882734856
LBFGS Accumulation History: 3 points
Removed measurement 3e76bd23 to history. Total: 3
th(0)=-1648.7141133465275;dx=-0.0722370867350843
Armijo: th(30296.737828573387)=-1343.7936702865568; dx=0.008006354834798269 evalInputDelta=-304.9204430599707
New Minimum: -1648.7141133465275 > -1768.9087798015505
END: th(15148.368914286693)=-1768.9087798015505; dx=-0.004572600798647036 evalInputDelta=120.19466645502303
Fitness changed from -1648.7141133465275 to -1768.9087798015505
Iteration 13 complete. Error: -1768.9087798015505 Total: 2.6991; Orientation: 0.2909; Line Search: 1.8413
Adding measurement 931bb51 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.733e+05, gradient 2.718e-01, dot -0.204; [950ae0c5-0b4c-4368-8d49-fef1bb6f05c7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1768.9087798015505, -378.960802479516, -269.6965385945564, -193.82148882734856
LBFGS Accumulation History: 3 points
Removed measurement 931bb51 to history. Total: 3
th(0)=-1768.9087798015505;dx=-0.07380640949110437
Armijo: th(32636.171486339877)=-1427.7355484380237; dx=0.008147252728338715 evalInputDelta=-341.17323136352684
New Minimum: -1768.9087798015505 > -1890.2133527232693
END: th(16318.085743169939)=-1890.2133527232693; dx=-0.003655544715018534 evalInputDelta=121.30457292171877
Fitness changed from -1768.9087798015505 to -1890.2133527232693
Iteration 14 complete. Error: -1890.2133527232693 Total: 2.6270; Orientation: 0.2932; Line Search: 1.7944
Adding measurement cdf7f2c to history. Total: 3
Accepted: LBFGS Orientation magnitude: 2.561e+04, gradient 2.711e-01, dot 0.276; [950ae0c5-0b4c-4368-8d49-fef1bb6f05c7 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-1890.2133527232693;dx=-1914.335629399528
Armijo: th(35156.25000000003)=4.868524465818382; dx=0.030499084473727373 evalInputDelta=-1895.0818771890877
Armijo: th(17578.125000000015)=3.2905436331130886; dx=0.061189665591350056 evalInputDelta=-1893.5038963563825
Armijo: th(5859.375000000005)=-1.392693516749432; dx=0.18263570283852826 evalInputDelta=-1888.8206592065198
Armijo: th(1464.8437500000011)=-13.136045044051343; dx=0.7230176283607375 evalInputDelta=-1877.077307679218
Armijo: th(292.9687500000002)=-45.160069373824854; dx=3.5129007814160205 evalInputDelta=-1845.0532833494444
Armijo: th(48.828125000000036)=-154.53776460179222; dx=18.73756679744081 evalInputDelta=-1735.6755881214772
Armijo: th(6.975446428571433)=-562.5262845904483; dx=85.13975087619839 evalInputDelta=-1327.687068132821
Armijo: th(0.8719308035714292)=-1779.381798688023; dx=20.26303106791231 evalInputDelta=-110.8315540352462
New Minimum: -1890.2133527232693 > -1987.6936565330104
END: th(0.09688120039682546)=-1987.6936565330104; dx=-945.6110301678511 evalInputDelta=97.48030380974114
Fitness changed from -1890.2133527232693 to -1987.6936565330104
Iteration 15 complete. Error: -1987.6936565330104 Total: 7.2262; Orientation: 0.3082; Line Search: 6.3464
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -1987.6936565330104 (> -Infinity) after 47.385s (< 5400.000s)
Returns
-1987.6936565330104