BasicOptimizer.scala:89 executed in 47.25 seconds (1.556 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 224537208295100
Reset training subject: 224537862947900
Adding measurement 4e3ca3e6 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-147.32539320963986;dx=-0.1095501943774066
New Minimum: -147.32539320963986 > -147.4679782527878
WOLFE (weak): th(2.154434690031884)=-147.4679782527878; dx=-0.10925394285416984 evalInputDelta=0.14258504314793186
New Minimum: -147.4679782527878 > -147.61023979880105
WOLFE (weak): th(4.308869380063768)=-147.61023979880105; dx=-0.1088507705016862 evalInputDelta=0.2848465891611909
New Minimum: -147.61023979880105 > -148.17670898676926
WOLFE (weak): th(12.926608140191302)=-148.17670898676926; dx=-0.10749662095821516 evalInputDelta=0.8513157771294004
New Minimum: -148.17670898676926 > -150.66239482974424
WOLFE (weak): th(51.70643256076521)=-150.66239482974424; dx=-0.10331743833955871 evalInputDelta=3.337001620104388
New Minimum: -150.66239482974424 > -162.3160377207384
END: th(258.53216280382605)=-162.3160377207384; dx=-0.0850920209077555 evalInputDelta=14.990644511098537
Fitness changed from -147.32539320963986 to -162.3160377207384
Iteration 1 complete. Error: -162.3160377207384 Total: 5.7044; Orientation: 0.0165; Line Search: 3.8382
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement c220e27 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-162.3160377207384;dx=-0.12558482094336504
New Minimum: -162.3160377207384 > -192.3626596244397
END: th(556.9906600335335)=-192.3626596244397; dx=-0.07450814707868184 evalInputDelta=30.046621903701293
Fitness changed from -162.3160377207384 to -192.3626596244397
Iteration 2 complete. Error: -192.3626596244397 Total: 1.8508; Orientation: 0.0166; Line Search: 1.2398
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 2837586 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-192.3626596244397;dx=-0.12771172008833787
New Minimum: -192.3626596244397 > -272.09731619969784
END: th(1200.0)=-272.09731619969784; dx=-0.07166104858893706 evalInputDelta=79.73465657525816
Fitness changed from -192.3626596244397 to -272.09731619969784
Iteration 3 complete. Error: -272.09731619969784 Total: 2.1350; Orientation: 0.0174; Line Search: 1.4913
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 5233bc20 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.685e+03, gradient 3.698e-01, dot -0.547; [5f067a88-7d3d-461b-a9c9-5adf7f353840 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -272.09731619969784, -192.3626596244397, -162.3160377207384, -147.32539320963986
LBFGS Accumulation History: 3 points
Removed measurement 5233bc20 to history. Total: 3
th(0)=-272.09731619969784;dx=-0.13640093829143674
New Minimum: -272.09731619969784 > -417.11694724491997
END: th(2585.3216280382608)=-417.11694724491997; dx=-0.05062899582915199 evalInputDelta=145.01963104522213
Fitness changed from -272.09731619969784 to -417.11694724491997
Iteration 4 complete. Error: -417.11694724491997 Total: 2.1970; Orientation: 0.3090; Line Search: 1.2884
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 4517c401 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.530e+03, gradient 3.715e-01, dot -0.329; [5f067a88-7d3d-461b-a9c9-5adf7f353840 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -417.11694724491997, -192.3626596244397, -162.3160377207384, -147.32539320963986
LBFGS Accumulation History: 3 points
Removed measurement 4517c401 to history. Total: 3
th(0)=-417.11694724491997;dx=-0.1377523751132847
New Minimum: -417.11694724491997 > -628.7899691559197
END: th(5569.9066003353355)=-628.7899691559197; dx=-0.02817771014856499 evalInputDelta=211.67302191099975
Fitness changed from -417.11694724491997 to -628.7899691559197
Iteration 5 complete. Error: -628.7899691559197 Total: 2.1588; Orientation: 0.3144; Line Search: 1.2378
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 61fa05b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.669e+03, gradient 3.883e-01, dot -0.194; [5f067a88-7d3d-461b-a9c9-5adf7f353840 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -628.7899691559197, -192.3626596244397, -162.3160377207384, -147.32539320963986
LBFGS Accumulation History: 3 points
Removed measurement 61fa05b to history. Total: 3
th(0)=-628.7899691559197;dx=-0.15054288204747446
New Minimum: -628.7899691559197 > -835.615994716279
END: th(12000.000000000002)=-835.615994716279; dx=-0.010910220473994054 evalInputDelta=206.8260255603593
Fitness changed from -628.7899691559197 to -835.615994716279
Iteration 6 complete. Error: -835.615994716279 Total: 2.1208; Orientation: 0.3066; Line Search: 1.2253
Adding measurement 17a9b784 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.667e+03, gradient 3.805e-01, dot -0.121; [5f067a88-7d3d-461b-a9c9-5adf7f353840 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -835.615994716279, -192.3626596244397, -162.3160377207384, -147.32539320963986
LBFGS Accumulation History: 3 points
Removed measurement 17a9b784 to history. Total: 3
th(0)=-835.615994716279;dx=-0.14443773094324458
New Minimum: -835.615994716279 > -901.0867705182818
END: th(25853.21628038261)=-901.0867705182818; dx=-0.0029463117235238585 evalInputDelta=65.47077580200278
Fitness changed from -835.615994716279 to -901.0867705182818
Iteration 7 complete. Error: -901.0867705182818 Total: 2.1669; Orientation: 0.3137; Line Search: 1.2585
Adding measurement 33cbcca6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.566e+03, gradient 3.387e-01, dot -0.078; [5f067a88-7d3d-461b-a9c9-5adf7f353840 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -901.0867705182818, -192.3626596244397, -162.3160377207384, -147.32539320963986
LBFGS Accumulation History: 3 points
Removed measurement 33cbcca6 to history. Total: 3
th(0)=-901.0867705182818;dx=-0.1144142165727684
New Minimum: -901.0867705182818 > -919.6066819764754
END: th(55699.066003353364)=-919.6066819764754; dx=-0.0010377389124246938 evalInputDelta=18.519911458193633
Fitness changed from -901.0867705182818 to -919.6066819764754
Iteration 8 complete. Error: -919.6066819764754 Total: 2.1527; Orientation: 0.3107; Line Search: 1.2722
Adding measurement 414ff5d3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.236e+04, gradient 2.871e-01, dot -0.061; [5f067a88-7d3d-461b-a9c9-5adf7f353840 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -919.6066819764754, -192.3626596244397, -162.3160377207384, -147.32539320963986
LBFGS Accumulation History: 3 points
Removed measurement 414ff5d3 to history. Total: 3
th(0)=-919.6066819764754;dx=-0.08222191157205246
Armijo: th(120000.00000000004)=-844.1403560637337; dx=2.1781034983275676E-4 evalInputDelta=-75.4663259127417
New Minimum: -919.6066819764754 > -1477.8554406550754
END: th(60000.00000000002)=-1477.8554406550754; dx=-0.006671388812943211 evalInputDelta=558.2487586786
Fitness changed from -919.6066819764754 to -1477.8554406550754
Iteration 9 complete. Error: -1477.8554406550754 Total: 3.5741; Orientation: 0.2923; Line Search: 1.8262
Adding measurement 10f8d6e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.233e+04, gradient 2.733e-01, dot -0.089; [5f067a88-7d3d-461b-a9c9-5adf7f353840 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1477.8554406550754, -192.3626596244397, -162.3160377207384, -147.32539320963986
LBFGS Accumulation History: 3 points
Removed measurement 10f8d6e to history. Total: 3
th(0)=-1477.8554406550754;dx=-0.07463143640154951
Armijo: th(129266.08140191308)=-1088.325753218776; dx=0.0012295327253645017 evalInputDelta=-389.52968743629935
New Minimum: -1477.8554406550754 > -2078.2404328876464
END: th(64633.04070095654)=-2078.2404328876464; dx=-0.008292875024093455 evalInputDelta=600.384992232571
Fitness changed from -1477.8554406550754 to -2078.2404328876464
Iteration 10 complete. Error: -2078.2404328876464 Total: 2.7165; Orientation: 0.3083; Line Search: 1.8503
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 6f197810 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.972e+08, gradient 2.802e-01, dot -0.119; [5f067a88-7d3d-461b-a9c9-5adf7f353840 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2078.2404328876464, -192.3626596244397, -162.3160377207384, -147.32539320963986
LBFGS Accumulation History: 3 points
Removed measurement 6f197810 to history. Total: 3
th(0)=-2078.2404328876464;dx=-0.07827978774032916
Armijo: th(139247.66500838342)=-1204.911800408795; dx=0.002371501190770742 evalInputDelta=-873.3286324788514
New Minimum: -2078.2404328876464 > -2462.2124596460326
END: th(69623.83250419171)=-2462.2124596460326; dx=-0.00689480207145352 evalInputDelta=383.97202675838616
Fitness changed from -2078.2404328876464 to -2462.2124596460326
Iteration 11 complete. Error: -2462.2124596460326 Total: 2.7684; Orientation: 0.3087; Line Search: 1.8584
Adding measurement 48c4b127 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.040e+05, gradient 2.911e-01, dot 0.134; [5f067a88-7d3d-461b-a9c9-5adf7f353840 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-2462.2124596460326;dx=-4063.307137840935
Armijo: th(150000.00000000006)=-622.87571029379; dx=0.006449612881529207 evalInputDelta=-1839.3367493522426
Armijo: th(75000.00000000003)=-624.1084923767612; dx=0.012890625693008029 evalInputDelta=-1838.1039672692714
Armijo: th(25000.00000000001)=-626.4444127589354; dx=0.03866681178122433 evalInputDelta=-1835.7680468870972
Armijo: th(6250.000000000003)=-631.237001454992; dx=0.1539328052330804 evalInputDelta=-1830.9754581910406
Armijo: th(1250.0000000000005)=-641.4771860956521; dx=0.7655236113100422 evalInputDelta=-1820.7352735503805
Armijo: th(208.3333333333334)=-661.5824172866126; dx=4.51813992996723 evalInputDelta=-1800.63004235942
Armijo: th(29.76190476190477)=-706.563713270154; dx=30.202153485633083 evalInputDelta=-1755.6487463758785
Armijo: th(3.7202380952380962)=-912.8111585510828; dx=187.6422481401903 evalInputDelta=-1549.4013010949498
Armijo: th(0.4133597883597885)=-1786.5890968561762; dx=473.33316404758654 evalInputDelta=-675.6233627898564
Armijo: th(0.04133597883597885)=-2415.7932126871096; dx=-550.0114405050798 evalInputDelta=-46.41924695892294
Armijo: th(0.003757816257816259)=-2459.1674772362903; dx=-961.7219652062327 evalInputDelta=-3.0449824097422606
Armijo: th(3.131513548180216E-4)=-2461.9656385595804; dx=-993.2026665261066 evalInputDelta=-0.2468210864522007
Armijo: th(2.408856575523243E-5)=-2462.1937605974917; dx=-995.5927249390811 evalInputDelta=-0.018699048540838703
Armijo: th(1.7206118396594594E-6)=-2462.2114773614867; dx=-995.9868805408296 evalInputDelta=-9.822845458984375E-4
Armijo: th(1.1470745597729729E-7)=-2462.2123833520873; dx=-993.2799708165771 evalInputDelta=-7.62939453125E-5
New Minimum: -2462.2124596460326 > -2462.212746722672
END: th(7.169215998581081E-9)=-2462.212746722672; dx=-997.0194379155129 evalInputDelta=2.8707663932436844E-4
Fitness changed from -2462.2124596460326 to -2462.212746722672
Iteration 12 complete. Error: -2462.212746722672 Total: 12.2149; Orientation: 0.3058; Line Search: 11.3385
Adding measurement 5c852b69 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 4.137e+06, gradient 2.911e-01, dot -0.109; [5f067a88-7d3d-461b-a9c9-5adf7f353840 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2462.212746722672, -2462.2124596460326, -192.3626596244397, -162.3160377207384, -147.32539320963986
Rejected: LBFGS Orientation magnitude: 1.166e+07, gradient 2.911e-01, dot -0.110; [5f067a88-7d3d-461b-a9c9-5adf7f353840 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2462.212746722672, -2462.2124596460326, -192.3626596244397, -162.3160377207384
LBFGS Accumulation History: 3 points
Removed measurement 5c852b69 to history. Total: 4
Removed measurement 48c4b127 to history. Total: 3
th(0)=-2462.212746722672;dx=-0.08460221264838684
END: th(1.5445607647674653E-8)=-2462.212746722672; dx=-0.0675811905327225 evalInputDelta=0.0
Fitness changed from -2462.212746722672 to -2462.212746722672
Static Iteration Total: 2.9917; Orientation: 0.7053; Line Search: 1.6549
Iteration 13 failed. Error: -2462.212746722672
Previous Error: 0.0 -> -2462.212746722672
Retrying iteration 13
Reset training subject: 224581960396200
Adding measurement 3602b500 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-2462.212746722672;dx=-0.08455911710506647
END: th(3.327655292477203E-8)=-2462.212746722672; dx=-0.0675083307883787 evalInputDelta=0.0
Fitness changed from -2462.212746722672 to -2462.212746722672
Static Iteration Total: 2.4982; Orientation: 0.0153; Line Search: 1.3201
Iteration 14 failed. Error: -2462.212746722672
Previous Error: 0.0 -> -2462.212746722672
Optimization terminated 14
Final threshold in iteration 14: -2462.212746722672 (> -Infinity) after 47.250s (< 5400.000s)
Returns
-2462.212746722672