BasicOptimizer.scala:89 executed in 8.71 seconds (0.172 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 160117639153900
Reset training subject: 160117766817700
Adding measurement 3def1ecc to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-14.669364738829705;dx=-0.04865428392932537
New Minimum: -14.669364738829705 > -14.799176497359575
WOLFE (weak): th(2.154434690031884)=-14.799176497359575; dx=-0.047646249337395405 evalInputDelta=0.12981175852987015
New Minimum: -14.799176497359575 > -14.927949925051456
WOLFE (weak): th(4.308869380063768)=-14.927949925051456; dx=-0.04855809640509435 evalInputDelta=0.258585186221751
New Minimum: -14.927949925051456 > -15.442576697197529
WOLFE (weak): th(12.926608140191302)=-15.442576697197529; dx=-0.04789918319315352 evalInputDelta=0.7732119583678241
New Minimum: -15.442576697197529 > -17.84029407070682
WOLFE (weak): th(51.70643256076521)=-17.84029407070682; dx=-0.05065627910665273 evalInputDelta=3.1709293318771135
New Minimum: -17.84029407070682 > -28.68622980235356
END: th(258.53216280382605)=-28.68622980235356; dx=-0.03476840597036641 evalInputDelta=14.016865063523854
Fitness changed from -14.669364738829705 to -28.68622980235356
Iteration 1 complete. Error: -28.68622980235356 Total: 1.0241; Orientation: 0.0014; Line Search: 0.6731
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 8e241a4 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-28.68622980235356;dx=-0.06346116724048403
New Minimum: -28.68622980235356 > -40.242631131072464
END: th(556.9906600335335)=-40.242631131072464; dx=-0.017162060469914912 evalInputDelta=11.556401328718906
Fitness changed from -28.68622980235356 to -40.242631131072464
Iteration 2 complete. Error: -40.242631131072464 Total: 0.5718; Orientation: 0.0016; Line Search: 0.4414
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 5ab8b340 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-40.242631131072464;dx=-0.09898415173403
New Minimum: -40.242631131072464 > -58.17684599697933
END: th(1200.0)=-58.17684599697933; dx=-0.004205820901768102 evalInputDelta=17.934214865906867
Fitness changed from -40.242631131072464 to -58.17684599697933
Iteration 3 complete. Error: -58.17684599697933 Total: 0.3463; Orientation: 0.0016; Line Search: 0.2238
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 25368c34 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.417e+03, gradient 3.137e-01, dot -0.185; [2d0e9195-5629-4fc0-98b7-28942458c3f8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -58.17684599697933, -40.242631131072464, -28.68622980235356, -14.669364738829705
LBFGS Accumulation History: 3 points
Removed measurement 25368c34 to history. Total: 3
th(0)=-58.17684599697933;dx=-0.09353355460913113
New Minimum: -58.17684599697933 > -97.53543526745806
WOLF (strong): th(2585.3216280382608)=-97.53543526745806; dx=2.1560133492815904E-4 evalInputDelta=39.358589270478724
New Minimum: -97.53543526745806 > -97.7039533488794
WOLF (strong): th(1292.6608140191304)=-97.7039533488794; dx=4.525157645577689E-4 evalInputDelta=39.527107351900064
END: th(430.8869380063768)=-96.54608982728146; dx=-0.04566532193936559 evalInputDelta=38.36924383030213
Fitness changed from -58.17684599697933 to -97.7039533488794
Iteration 4 complete. Error: -97.7039533488794 Total: 0.6110; Orientation: 0.0344; Line Search: 0.4517
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 25c08abe to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.006e+03, gradient 3.893e-01, dot -0.181; [2d0e9195-5629-4fc0-98b7-28942458c3f8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -97.7039533488794, -40.242631131072464, -28.68622980235356, -14.669364738829705
LBFGS Accumulation History: 3 points
Removed measurement 25c08abe to history. Total: 3
th(0)=-97.7039533488794;dx=-0.14772544194314466
New Minimum: -97.7039533488794 > -170.76781454470571
END: th(928.3177667225559)=-170.76781454470571; dx=-0.005027202731260213 evalInputDelta=73.06386119582632
Fitness changed from -97.7039533488794 to -170.76781454470571
Iteration 5 complete. Error: -170.76781454470571 Total: 0.3722; Orientation: 0.0272; Line Search: 0.2256
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 118bc729 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.316e+03, gradient 3.888e-01, dot -0.154; [2d0e9195-5629-4fc0-98b7-28942458c3f8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -170.76781454470571, -40.242631131072464, -28.68622980235356, -14.669364738829705
LBFGS Accumulation History: 3 points
Removed measurement 118bc729 to history. Total: 3
th(0)=-170.76781454470571;dx=-0.14906480459784885
New Minimum: -170.76781454470571 > -188.1572231539358
WOLF (strong): th(2000.0000000000005)=-188.1572231539358; dx=0.02334683769764318 evalInputDelta=17.389408609230088
New Minimum: -188.1572231539358 > -227.72669851984855
END: th(1000.0000000000002)=-227.72669851984855; dx=-1.8007012059748952E-4 evalInputDelta=56.95888397514284
Fitness changed from -170.76781454470571 to -227.72669851984855
Iteration 6 complete. Error: -227.72669851984855 Total: 0.4894; Orientation: 0.0284; Line Search: 0.3436
Adding measurement 3b7c7b48 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.347e+03, gradient 3.517e-01, dot -0.214; [2d0e9195-5629-4fc0-98b7-28942458c3f8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -227.72669851984855, -40.242631131072464, -28.68622980235356, -14.669364738829705
LBFGS Accumulation History: 3 points
Removed measurement 3b7c7b48 to history. Total: 3
th(0)=-227.72669851984855;dx=-0.11984143026048903
New Minimum: -227.72669851984855 > -231.9105430979799
WOLF (strong): th(2154.434690031884)=-231.9105430979799; dx=0.010533048430566995 evalInputDelta=4.183844578131357
New Minimum: -231.9105430979799 > -265.1533901584138
WOLF (strong): th(1077.217345015942)=-265.1533901584138; dx=0.0018461578130726268 evalInputDelta=37.426691638565245
New Minimum: -265.1533901584138 > -265.3154372628928
END: th(359.07244833864735)=-265.3154372628928; dx=-0.059375344797171076 evalInputDelta=37.58873874304422
Fitness changed from -227.72669851984855 to -265.3154372628928
Iteration 7 complete. Error: -265.3154372628928 Total: 0.5884; Orientation: 0.0276; Line Search: 0.4535
Adding measurement 29dd418b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.569e+03, gradient 2.868e-01, dot -0.235; [2d0e9195-5629-4fc0-98b7-28942458c3f8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -265.3154372628928, -40.242631131072464, -28.68622980235356, -14.669364738829705
LBFGS Accumulation History: 3 points
Removed measurement 29dd418b to history. Total: 3
th(0)=-265.3154372628928;dx=-0.07891811489550851
New Minimum: -265.3154372628928 > -303.0194790562151
END: th(773.5981389354633)=-303.0194790562151; dx=-0.033256248254535145 evalInputDelta=37.70404179332235
Fitness changed from -265.3154372628928 to -303.0194790562151
Iteration 8 complete. Error: -303.0194790562151 Total: 0.3792; Orientation: 0.0271; Line Search: 0.2380
Adding measurement 7d036190 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.508e+04, gradient 3.133e-01, dot -0.293; [2d0e9195-5629-4fc0-98b7-28942458c3f8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -303.0194790562151, -40.242631131072464, -28.68622980235356, -14.669364738829705
LBFGS Accumulation History: 3 points
Removed measurement 7d036190 to history. Total: 3
th(0)=-303.0194790562151;dx=-0.09558099026732994
New Minimum: -303.0194790562151 > -313.6806571778056
END: th(1666.6666666666672)=-313.6806571778056; dx=-0.0023011303906388114 evalInputDelta=10.661178121590467
Fitness changed from -303.0194790562151 to -313.6806571778056
Iteration 9 complete. Error: -313.6806571778056 Total: 0.3656; Orientation: 0.0310; Line Search: 0.2264
Adding measurement 4da6989f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.712e+04, gradient 4.031e-01, dot -0.224; [2d0e9195-5629-4fc0-98b7-28942458c3f8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -313.6806571778056, -40.242631131072464, -28.68622980235356, -14.669364738829705
LBFGS Accumulation History: 3 points
Removed measurement 4da6989f to history. Total: 3
th(0)=-313.6806571778056;dx=-0.15965337851306347
Armijo: th(3590.7244833864743)=-225.37528598151573; dx=0.017340811707910297 evalInputDelta=-88.30537119628985
New Minimum: -313.6806571778056 > -319.35344839221636
WOLF (strong): th(1795.3622416932371)=-319.35344839221636; dx=0.02398509640438011 evalInputDelta=5.672791214410779
New Minimum: -319.35344839221636 > -379.2930512809324
END: th(598.4540805644124)=-379.2930512809324; dx=-0.03299499572295296 evalInputDelta=65.6123941031268
Fitness changed from -313.6806571778056 to -379.2930512809324
Iteration 10 complete. Error: -379.2930512809324 Total: 0.6066; Orientation: 0.0265; Line Search: 0.4541
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 7f7492aa to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.393e+05, gradient 3.299e-01, dot -0.293; [2d0e9195-5629-4fc0-98b7-28942458c3f8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -379.2930512809324, -40.242631131072464, -28.68622980235356, -14.669364738829705
LBFGS Accumulation History: 3 points
Removed measurement 7f7492aa to history. Total: 3
th(0)=-379.2930512809324;dx=-0.10549994655405584
New Minimum: -379.2930512809324 > -391.47054486895945
END: th(1289.3302315591059)=-391.47054486895945; dx=-0.0024592177387636853 evalInputDelta=12.177493588027062
Fitness changed from -379.2930512809324 to -391.47054486895945
Iteration 11 complete. Error: -391.47054486895945 Total: 0.3815; Orientation: 0.0291; Line Search: 0.2316
Adding measurement 11aa8ba6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.666e+04, gradient 4.228e-01, dot -0.211; [2d0e9195-5629-4fc0-98b7-28942458c3f8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -391.47054486895945, -40.242631131072464, -28.68622980235356, -14.669364738829705
LBFGS Accumulation History: 3 points
Removed measurement 11aa8ba6 to history. Total: 3
th(0)=-391.47054486895945;dx=-0.1756420025097619
Armijo: th(2777.777777777779)=-279.0270064214215; dx=0.02690650377052775 evalInputDelta=-112.44353844753795
Armijo: th(1388.8888888888896)=-380.3833797959632; dx=0.044347174274322936 evalInputDelta=-11.087165072996243
New Minimum: -391.47054486895945 > -453.4707474027404
END: th(462.9629629629632)=-453.4707474027404; dx=-0.030057625355595315 evalInputDelta=62.000202533780964
Fitness changed from -391.47054486895945 to -453.4707474027404
Iteration 12 complete. Error: -453.4707474027404 Total: 0.6089; Orientation: 0.0274; Line Search: 0.4707
Adding measurement 2aaf0936 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.220e+04, gradient 3.023e-01, dot 0.382; [2d0e9195-5629-4fc0-98b7-28942458c3f8 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-453.4707474027404;dx=-1425.414468119009
Armijo: th(997.4234676073542)=-195.8181654143711; dx=0.05840043521123109 evalInputDelta=-257.6525819883693
Armijo: th(498.7117338036771)=-195.49417550466816; dx=0.1144758782983796 evalInputDelta=-257.9765718980723
Armijo: th(166.23724460122568)=-195.3922337248848; dx=0.33821046846334524 evalInputDelta=-258.07851367785565
Armijo: th(41.55931115030642)=-199.6286102522632; dx=1.5072118077317818 evalInputDelta=-253.8421371504772
Armijo: th(8.311862230061283)=-197.67540668037083; dx=9.061929602151352 evalInputDelta=-255.7953407223696
Armijo: th(1.3853103716768806)=-220.04861856246677; dx=57.17704139835632 evalInputDelta=-233.42212884027364
New Minimum: -453.4707474027404 > -517.1601238978118
END: th(0.1979014816681258)=-517.1601238978118; dx=-110.81532809221167 evalInputDelta=63.68937649507137
Fitness changed from -453.4707474027404 to -517.1601238978118
Iteration 13 complete. Error: -517.1601238978118 Total: 1.0401; Orientation: 0.0277; Line Search: 0.9006
Adding measurement 1db25ca0 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.286e+03, gradient 3.816e-01, dot -0.333; [2d0e9195-5629-4fc0-98b7-28942458c3f8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -517.1601238978118, -453.4707474027404, -40.242631131072464, -28.68622980235356, -14.669364738829705
Rejected: LBFGS Orientation magnitude: 7.119e+03, gradient 3.816e-01, dot -0.166; [2d0e9195-5629-4fc0-98b7-28942458c3f8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -517.1601238978118, -453.4707474027404, -40.242631131072464, -28.68622980235356
LBFGS Accumulation History: 3 points
Removed measurement 1db25ca0 to history. Total: 4
Removed measurement 2aaf0936 to history. Total: 3
th(0)=-517.1601238978118;dx=-0.14201126189858815
New Minimum: -517.1601238978118 > -517.2287332505804
WOLFE (weak): th(0.4263658173145191)=-517.2287332505804; dx=-0.132760936204515 evalInputDelta=0.06860935276858982
New Minimum: -517.2287332505804 > -517.297363708728
WOLFE (weak): th(0.8527316346290382)=-517.297363708728; dx=-0.13249632818736193 evalInputDelta=0.13723981091618498
New Minimum: -517.297363708728 > -517.5718851038955
WOLFE (weak): th(2.558194903887115)=-517.5718851038955; dx=-0.13168097235043583 evalInputDelta=0.4117612060837246
New Minimum: -517.5718851038955 > -518.8042441518766
WOLFE (weak): th(10.23277961554846)=-518.8042441518766; dx=-0.1314015935216079 evalInputDelta=1.6441202540647737
New Minimum: -518.8042441518766 > -525.2680268205937
WOLFE (weak): th(51.163898077742296)=-525.2680268205937; dx=-0.12921344020253545 evalInputDelta=8.107902922781932
New Minimum: -525.2680268205937 > -562.1367160220012
END: th(306.9833884664538)=-562.1367160220012; dx=-0.11343276117523857 evalInputDelta=44.97659212418944
Fitness changed from -517.1601238978118 to -562.1367160220012
Iteration 14 complete. Error: -562.1367160220012 Total: 0.9515; Orientation: 0.0584; Line Search: 0.7845
Adding measurement 22f03a46 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.182e+03, gradient 3.426e-01, dot -0.227; [2d0e9195-5629-4fc0-98b7-28942458c3f8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -562.1367160220012, -40.242631131072464, -28.68622980235356, -14.669364738829705
LBFGS Accumulation History: 3 points
Removed measurement 22f03a46 to history. Total: 3
th(0)=-562.1367160220012;dx=-0.11357919393682639
New Minimum: -562.1367160220012 > -630.5648013884061
END: th(661.3756613756617)=-630.5648013884061; dx=-0.08058562488033011 evalInputDelta=68.42808536640484
Fitness changed from -562.1367160220012 to -630.5648013884061
Iteration 15 complete. Error: -630.5648013884061 Total: 0.3585; Orientation: 0.0271; Line Search: 0.2221
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -630.5648013884061 (> -Infinity) after 8.706s (< 5400.000s)
Returns
-630.5648013884061