BasicOptimizer.scala:89 executed in 8.82 seconds (0.464 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 153460828789300
Reset training subject: 153460959991800
Adding measurement 15b873b2 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-6.972103033978451;dx=-0.011161561482374504
New Minimum: -6.972103033978451 > -7.002491997609503
WOLFE (weak): th(2.154434690031884)=-7.002491997609503; dx=-0.011343149181038617 evalInputDelta=0.030388963631051702
New Minimum: -7.002491997609503 > -7.032762535532188
WOLFE (weak): th(4.308869380063768)=-7.032762535532188; dx=-0.010576197763333947 evalInputDelta=0.06065950155373745
New Minimum: -7.032762535532188 > -7.153775698867916
WOLFE (weak): th(12.926608140191302)=-7.153775698867916; dx=-0.010187265337787653 evalInputDelta=0.1816726648894651
New Minimum: -7.153775698867916 > -7.71150406412852
WOLFE (weak): th(51.70643256076521)=-7.71150406412852; dx=-0.01152856054961147 evalInputDelta=0.7394010301500691
New Minimum: -7.71150406412852 > -10.818352674800648
WOLFE (weak): th(258.53216280382605)=-10.818352674800648; dx=-0.011260876312397923 evalInputDelta=3.8462496408221973
New Minimum: -10.818352674800648 > -25.401392219335982
END: th(1551.1929768229563)=-25.401392219335982; dx=-0.0037044880778601247 evalInputDelta=18.429289185357533
Fitness changed from -6.972103033978451 to -25.401392219335982
Iteration 1 complete. Error: -25.401392219335982 Total: 1.1581; Orientation: 0.0015; Line Search: 0.7931
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 44ce100d to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-25.401392219335982;dx=-0.0081310467638539
New Minimum: -25.401392219335982 > -44.37287071280346
END: th(3341.943960201201)=-44.37287071280346; dx=-0.0012781456368329367 evalInputDelta=18.97147849346748
Fitness changed from -25.401392219335982 to -44.37287071280346
Iteration 2 complete. Error: -44.37287071280346 Total: 0.3557; Orientation: 0.0014; Line Search: 0.2308
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 54be5826 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-44.37287071280346;dx=-0.005661086175202868
New Minimum: -44.37287071280346 > -57.429036905806505
END: th(7200.000000000001)=-57.429036905806505; dx=-4.562288052557095E-4 evalInputDelta=13.056166193003044
Fitness changed from -44.37287071280346 to -57.429036905806505
Iteration 3 complete. Error: -57.429036905806505 Total: 0.3471; Orientation: 0.0015; Line Search: 0.2269
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 44059f3d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.352e+02, gradient 8.014e-02, dot -0.413; [05a84f0c-614f-45a9-84c7-5a754d4d5d9e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -57.429036905806505, -44.37287071280346, -25.401392219335982, -6.972103033978451
LBFGS Accumulation History: 3 points
Removed measurement 44059f3d to history. Total: 3
th(0)=-57.429036905806505;dx=-0.006293731054027913
New Minimum: -57.429036905806505 > -77.36836270725503
END: th(15511.929768229566)=-77.36836270725503; dx=-4.311989084044451E-6 evalInputDelta=19.939325801448526
Fitness changed from -57.429036905806505 to -77.36836270725503
Iteration 4 complete. Error: -77.36836270725503 Total: 0.3815; Orientation: 0.0289; Line Search: 0.2328
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 6874061 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.958e+02, gradient 6.933e-02, dot -0.255; [05a84f0c-614f-45a9-84c7-5a754d4d5d9e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -77.36836270725503, -44.37287071280346, -25.401392219335982, -6.972103033978451
LBFGS Accumulation History: 3 points
Removed measurement 6874061 to history. Total: 3
th(0)=-77.36836270725503;dx=-0.0046460262998135925
New Minimum: -77.36836270725503 > -124.47933055085262
END: th(33419.43960201202)=-124.47933055085262; dx=-1.4641394945276039E-4 evalInputDelta=47.11096784359759
Fitness changed from -77.36836270725503 to -124.47933055085262
Iteration 5 complete. Error: -124.47933055085262 Total: 0.3831; Orientation: 0.0280; Line Search: 0.2303
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 144319fe to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.689e+03, gradient 6.190e-02, dot -0.220; [05a84f0c-614f-45a9-84c7-5a754d4d5d9e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -124.47933055085262, -44.37287071280346, -25.401392219335982, -6.972103033978451
LBFGS Accumulation History: 3 points
Removed measurement 144319fe to history. Total: 3
th(0)=-124.47933055085262;dx=-0.0037143433109874746
New Minimum: -124.47933055085262 > -151.90857688642086
WOLF (strong): th(72000.00000000003)=-151.90857688642086; dx=2.7627892913460007E-4 evalInputDelta=27.429246335568237
New Minimum: -151.90857688642086 > -180.28030585802324
WOLF (strong): th(36000.000000000015)=-180.28030585802324; dx=5.6987989511377365E-5 evalInputDelta=55.80097530717062
END: th(12000.000000000005)=-177.02586459997872; dx=-0.0016249202580177749 evalInputDelta=52.546534049126095
Fitness changed from -124.47933055085262 to -180.28030585802324
Iteration 6 complete. Error: -180.28030585802324 Total: 0.6075; Orientation: 0.0282; Line Search: 0.4569
Adding measurement 762ea718 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.306e+03, gradient 6.669e-02, dot -0.236; [05a84f0c-614f-45a9-84c7-5a754d4d5d9e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -180.28030585802324, -44.37287071280346, -25.401392219335982, -6.972103033978451
LBFGS Accumulation History: 3 points
Removed measurement 762ea718 to history. Total: 3
th(0)=-180.28030585802324;dx=-0.004324505470584679
New Minimum: -180.28030585802324 > -238.86462112464758
END: th(25853.216280382618)=-238.86462112464758; dx=-1.0478084388903809E-4 evalInputDelta=58.584315266624344
Fitness changed from -180.28030585802324 to -238.86462112464758
Iteration 7 complete. Error: -238.86462112464758 Total: 0.3707; Orientation: 0.0265; Line Search: 0.2304
Adding measurement 63a9672a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.775e+03, gradient 5.551e-02, dot -0.317; [05a84f0c-614f-45a9-84c7-5a754d4d5d9e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -238.86462112464758, -44.37287071280346, -25.401392219335982, -6.972103033978451
LBFGS Accumulation History: 3 points
Removed measurement 63a9672a to history. Total: 3
th(0)=-238.86462112464758;dx=-0.00296357732108026
Armijo: th(55699.06600335338)=-231.75719163597466; dx=4.4094301924816584E-4 evalInputDelta=-7.107429488672921
New Minimum: -238.86462112464758 > -273.597941140727
END: th(27849.53300167669)=-273.597941140727; dx=-7.13068882829989E-5 evalInputDelta=34.73332001607943
Fitness changed from -238.86462112464758 to -273.597941140727
Iteration 8 complete. Error: -273.597941140727 Total: 0.4891; Orientation: 0.0284; Line Search: 0.3498
Adding measurement b660e93 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.734e+04, gradient 6.134e-02, dot -0.385; [05a84f0c-614f-45a9-84c7-5a754d4d5d9e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -273.597941140727, -44.37287071280346, -25.401392219335982, -6.972103033978451
LBFGS Accumulation History: 3 points
Removed measurement b660e93 to history. Total: 3
th(0)=-273.597941140727;dx=-0.00365331257094803
Armijo: th(60000.00000000004)=-246.94617340862914; dx=4.474765959142497E-4 evalInputDelta=-26.65176773209788
New Minimum: -273.597941140727 > -285.97404962402544
WOLF (strong): th(30000.00000000002)=-285.97404962402544; dx=4.8324270520802516E-4 evalInputDelta=12.376108483298424
New Minimum: -285.97404962402544 > -300.0607792699637
END: th(10000.000000000005)=-300.0607792699637; dx=-0.0012286117679748707 evalInputDelta=26.462838129236673
Fitness changed from -273.597941140727 to -300.0607792699637
Iteration 9 complete. Error: -300.0607792699637 Total: 0.6133; Orientation: 0.0289; Line Search: 0.4692
Adding measurement 5ede219 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.318e+04, gradient 4.615e-02, dot -0.508; [05a84f0c-614f-45a9-84c7-5a754d4d5d9e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -300.0607792699637, -44.37287071280346, -25.401392219335982, -6.972103033978451
LBFGS Accumulation History: 3 points
Removed measurement 5ede219 to history. Total: 3
th(0)=-300.0607792699637;dx=-0.002041610756825538
New Minimum: -300.0607792699637 > -318.95913652258304
END: th(21544.34690031885)=-318.95913652258304; dx=-7.628650142960038E-4 evalInputDelta=18.89835725261935
Fitness changed from -300.0607792699637 to -318.95913652258304
Iteration 10 complete. Error: -318.95913652258304 Total: 0.8425; Orientation: 0.0272; Line Search: 0.7030
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 333d94ff to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.730e+06, gradient 5.189e-02, dot -0.502; [05a84f0c-614f-45a9-84c7-5a754d4d5d9e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -318.95913652258304, -44.37287071280346, -25.401392219335982, -6.972103033978451
LBFGS Accumulation History: 3 points
Removed measurement 333d94ff to history. Total: 3
th(0)=-318.95913652258304;dx=-0.002491996430334588
New Minimum: -318.95913652258304 > -319.7530010132545
WOLF (strong): th(46415.88833612782)=-319.7530010132545; dx=1.99598014971811E-4 evalInputDelta=0.7938644906714671
New Minimum: -319.7530010132545 > -333.53124858534716
END: th(23207.94416806391)=-333.53124858534716; dx=-2.4505410493630566E-4 evalInputDelta=14.57211206276412
Fitness changed from -318.95913652258304 to -333.53124858534716
Iteration 11 complete. Error: -333.53124858534716 Total: 0.5079; Orientation: 0.0297; Line Search: 0.3463
Adding measurement 2c5c5149 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.817e+04, gradient 5.367e-02, dot -0.410; [05a84f0c-614f-45a9-84c7-5a754d4d5d9e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -333.53124858534716, -44.37287071280346, -25.401392219335982, -6.972103033978451
LBFGS Accumulation History: 3 points
Removed measurement 2c5c5149 to history. Total: 3
th(0)=-333.53124858534716;dx=-0.0027780727308886147
Armijo: th(50000.00000000004)=-280.8475257002219; dx=6.097377035035473E-4 evalInputDelta=-52.683722885125235
New Minimum: -333.53124858534716 > -338.15163575334157
WOLF (strong): th(25000.00000000002)=-338.15163575334157; dx=5.466312509003345E-4 evalInputDelta=4.620387167994409
New Minimum: -338.15163575334157 > -349.5429510408979
END: th(8333.33333333334)=-349.5429510408979; dx=-0.0011111276764159354 evalInputDelta=16.011702455550733
Fitness changed from -333.53124858534716 to -349.5429510408979
Iteration 12 complete. Error: -349.5429510408979 Total: 0.5965; Orientation: 0.0299; Line Search: 0.4569
Adding measurement 5660e74b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.016e+07, gradient 4.361e-02, dot -0.568; [05a84f0c-614f-45a9-84c7-5a754d4d5d9e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -349.5429510408979, -44.37287071280346, -25.401392219335982, -6.972103033978451
LBFGS Accumulation History: 3 points
Removed measurement 5660e74b to history. Total: 3
th(0)=-349.5429510408979;dx=-0.0018035601216628244
New Minimum: -349.5429510408979 > -359.54881159462917
END: th(17953.62241693238)=-359.54881159462917; dx=-7.630865148904293E-4 evalInputDelta=10.005860553731281
Fitness changed from -349.5429510408979 to -359.54881159462917
Iteration 13 complete. Error: -359.54881159462917 Total: 0.3614; Orientation: 0.0272; Line Search: 0.2277
Adding measurement 3772b6a1 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 4.246e+04, gradient 4.297e-02, dot 0.618; [05a84f0c-614f-45a9-84c7-5a754d4d5d9e = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-359.54881159462917;dx=-1132.7430170563541
Armijo: th(38679.90694677319)=-309.5242926451667; dx=0.0028877874399861176 evalInputDelta=-50.02451894946245
Armijo: th(19339.953473386595)=-309.5242926451667; dx=0.005921071093482677 evalInputDelta=-50.02451894946245
Armijo: th(6446.651157795532)=-309.52944220606867; dx=0.016847628279116837 evalInputDelta=-50.019369388560506
Armijo: th(1611.662789448883)=-309.1784881717955; dx=0.07045597966162201 evalInputDelta=-50.37032342283368
Armijo: th(322.3325578897766)=-309.35917911614536; dx=0.3438135122353827 evalInputDelta=-50.189632478483816
Armijo: th(53.72209298162943)=-311.58131979974655; dx=2.060598174055781 evalInputDelta=-47.96749179488262
Armijo: th(7.674584711661347)=-317.15585177176496; dx=14.899937322918472 evalInputDelta=-42.39295982286421
Armijo: th(0.9593230889576684)=-347.6988628072633; dx=87.70841562033814 evalInputDelta=-11.84994878736586
New Minimum: -359.54881159462917 > -407.2543245896928
END: th(0.10659145432862982)=-407.2543245896928; dx=-196.47269273206672 evalInputDelta=47.70551299506366
Fitness changed from -359.54881159462917 to -407.2543245896928
Iteration 14 complete. Error: -407.2543245896928 Total: 1.3027; Orientation: 0.0265; Line Search: 1.1619
Adding measurement 25633e7d to history. Total: 4
Accepted: LBFGS Orientation magnitude: 3.117e+04, gradient 4.988e-02, dot 0.381; [05a84f0c-614f-45a9-84c7-5a754d4d5d9e = 1.000/1.000e+00]
th(0)=-407.2543245896928;dx=-593.4444665082744
Armijo: th(0.2296443268665493)=-402.448748311399; dx=25.62789024382446 evalInputDelta=-4.805576278293813
New Minimum: -407.2543245896928 > -410.21314276971674
END: th(0.11482216343327464)=-410.21314276971674; dx=-26.80053740365326 evalInputDelta=2.9588181800239113
Fitness changed from -407.2543245896928 to -410.21314276971674
Iteration 15 complete. Error: -410.21314276971674 Total: 0.4923; Orientation: 0.0343; Line Search: 0.3428
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -410.21314276971674 (> -Infinity) after 8.821s (< 5400.000s)
Returns
-410.21314276971674