BasicOptimizer.scala:89 executed in 11.00 seconds (0.651 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 453977695394700
Reset training subject: 453977862005300
Adding measurement 4a82e8b2 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-1.0179313046149736;dx=-1.2155583939953878E-6
New Minimum: -1.0179313046149736 > -1.017937999340795
WOLFE (weak): th(2.154434690031884)=-1.017937999340795; dx=-1.2155349758349212E-6 evalInputDelta=6.694725821487779E-6
New Minimum: -1.017937999340795 > -1.0179447934482229
WOLFE (weak): th(4.308869380063768)=-1.0179447934482229; dx=-1.212105226453153E-6 evalInputDelta=1.3488833249253318E-5
New Minimum: -1.0179447934482229 > -1.0179716048735057
WOLFE (weak): th(12.926608140191302)=-1.0179716048735057; dx=-1.195379892266524E-6 evalInputDelta=4.030025853207775E-5
New Minimum: -1.0179716048735057 > -1.018092661762939
WOLFE (weak): th(51.70643256076521)=-1.018092661762939; dx=-1.214465523432978E-6 evalInputDelta=1.6135714796550182E-4
New Minimum: -1.018092661762939 > -1.0187383315234546
WOLFE (weak): th(258.53216280382605)=-1.0187383315234546; dx=-1.2134686867121442E-6 evalInputDelta=8.070269084810011E-4
New Minimum: -1.0187383315234546 > -1.0227767455271015
WOLFE (weak): th(1551.1929768229563)=-1.0227767455271015; dx=-1.2153352036341504E-6 evalInputDelta=0.004845440912127907
New Minimum: -1.0227767455271015 > -1.0518443387341723
WOLFE (weak): th(10858.350837760694)=-1.0518443387341723; dx=-1.220487121986232E-6 evalInputDelta=0.03391303411919866
New Minimum: -1.0518443387341723 > -1.2800275997003472
WOLFE (weak): th(86866.80670208555)=-1.2800275997003472; dx=-1.1009972773822879E-6 evalInputDelta=0.2620962950853736
New Minimum: -1.2800275997003472 > -2.0541437602045227
END: th(781801.26031877)=-2.0541437602045227; dx=-9.8659884774839E-8 evalInputDelta=1.036212455589549
Fitness changed from -1.0179313046149736 to -2.0541437602045227
Iteration 1 complete. Error: -2.0541437602045227 Total: 1.8463; Orientation: 0.0028; Line Search: 1.3934
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 501668df to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-2.0541437602045227;dx=-1.162868471871769E-6
New Minimum: -2.0541437602045227 > -2.739148647481493
WOLF (strong): th(1684339.7559414052)=-2.739148647481493; dx=1.2307084275991602E-7 evalInputDelta=0.6850048872769703
New Minimum: -2.739148647481493 > -2.9726830221804086
END: th(842169.8779707026)=-2.9726830221804086; dx=-2.5784636693014603E-8 evalInputDelta=0.9185392619758859
Fitness changed from -2.0541437602045227 to -2.9726830221804086
Iteration 2 complete. Error: -2.9726830221804086 Total: 0.5868; Orientation: 0.0023; Line Search: 0.4340
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 1bd4529e to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-2.9726830221804086;dx=-1.0907376816134117E-6
New Minimum: -2.9726830221804086 > -3.4245614133312707
WOLF (strong): th(1814400.0)=-3.4245614133312707; dx=1.9720139321312968E-7 evalInputDelta=0.45187839115086215
New Minimum: -3.4245614133312707 > -3.846192425367368
WOLF (strong): th(907200.0)=-3.846192425367368; dx=6.258312950104665E-8 evalInputDelta=0.8735094031869592
END: th(302400.0)=-3.5848685992523515; dx=-5.820243500523176E-7 evalInputDelta=0.6121855770719429
Fitness changed from -2.9726830221804086 to -3.846192425367368
Iteration 3 complete. Error: -3.846192425367368 Total: 0.9895; Orientation: 0.0029; Line Search: 0.8171
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 60d1bf54 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.263e+03, gradient 1.001e-03, dot -0.535; [5a65d381-045d-40f1-8017-1c264e513100 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.846192425367368, -2.9726830221804086, -2.0541437602045227, -1.0179313046149736
LBFGS Accumulation History: 3 points
Removed measurement 60d1bf54 to history. Total: 3
th(0)=-3.846192425367368;dx=-9.853900774698649E-7
New Minimum: -3.846192425367368 > -4.652461593016245
END: th(651501.0502656417)=-4.652461593016245; dx=-1.3756229192734135E-7 evalInputDelta=0.8062691676488769
Fitness changed from -3.846192425367368 to -4.652461593016245
Iteration 4 complete. Error: -4.652461593016245 Total: 0.4818; Orientation: 0.0494; Line Search: 0.2785
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 118e2281 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.859e+03, gradient 7.713e-04, dot -0.625; [5a65d381-045d-40f1-8017-1c264e513100 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.652461593016245, -2.9726830221804086, -2.0541437602045227, -1.0179313046149736
LBFGS Accumulation History: 3 points
Removed measurement 118e2281 to history. Total: 3
th(0)=-4.652461593016245;dx=-5.903465625528571E-7
New Minimum: -4.652461593016245 > -5.347620517618384
END: th(1403616.4632845044)=-5.347620517618384; dx=-1.7303690036894553E-8 evalInputDelta=0.695158924602139
Fitness changed from -4.652461593016245 to -5.347620517618384
Iteration 5 complete. Error: -5.347620517618384 Total: 0.4769; Orientation: 0.0434; Line Search: 0.2872
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 4ddda69a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.077e+03, gradient 9.799e-04, dot -0.493; [5a65d381-045d-40f1-8017-1c264e513100 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.347620517618384, -2.9726830221804086, -2.0541437602045227, -1.0179313046149736
LBFGS Accumulation History: 3 points
Removed measurement 4ddda69a to history. Total: 3
th(0)=-5.347620517618384;dx=-9.545005154629524E-7
Armijo: th(3024000.0)=-5.115234930607438; dx=2.2595263310851546E-7 evalInputDelta=-0.23238558701094547
New Minimum: -5.347620517618384 > -6.082318293473513
WOLF (strong): th(1512000.0)=-6.082318293473513; dx=2.11932437449502E-7 evalInputDelta=0.7346977758551292
New Minimum: -6.082318293473513 > -6.138731732592362
END: th(504000.0)=-6.138731732592362; dx=-3.6952038968103887E-7 evalInputDelta=0.7911112149739781
Fitness changed from -5.347620517618384 to -6.138731732592362
Iteration 6 complete. Error: -6.138731732592362 Total: 0.7514; Orientation: 0.0434; Line Search: 0.5582
Adding measurement 6f65ab4e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.823e+03, gradient 6.693e-04, dot -0.697; [5a65d381-045d-40f1-8017-1c264e513100 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.138731732592362, -2.9726830221804086, -2.0541437602045227, -1.0179313046149736
LBFGS Accumulation History: 3 points
Removed measurement 6f65ab4e to history. Total: 3
th(0)=-6.138731732592362;dx=-4.456895100492754E-7
New Minimum: -6.138731732592362 > -6.890751912862264
END: th(1085835.0837760693)=-6.890751912862264; dx=-1.9705718737721303E-7 evalInputDelta=0.752020180269902
Fitness changed from -6.138731732592362 to -6.890751912862264
Iteration 7 complete. Error: -6.890751912862264 Total: 0.4635; Orientation: 0.0476; Line Search: 0.2777
Adding measurement 2648323a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.177e+04, gradient 7.170e-04, dot -0.660; [5a65d381-045d-40f1-8017-1c264e513100 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.890751912862264, -2.9726830221804086, -2.0541437602045227, -1.0179313046149736
LBFGS Accumulation History: 3 points
Removed measurement 2648323a to history. Total: 3
th(0)=-6.890751912862264;dx=-5.097989817207166E-7
New Minimum: -6.890751912862264 > -7.751384676473903
WOLF (strong): th(2339360.7721408405)=-7.751384676473903; dx=5.840630390726082E-10 evalInputDelta=0.8606327636116395
END: th(1169680.3860704203)=-7.630168791921426; dx=-1.0090436181494618E-7 evalInputDelta=0.7394168790591618
Fitness changed from -6.890751912862264 to -7.751384676473903
Iteration 8 complete. Error: -7.751384676473903 Total: 0.5881; Orientation: 0.0454; Line Search: 0.4106
Adding measurement 1322afc6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.165e+04, gradient 9.966e-04, dot -0.435; [5a65d381-045d-40f1-8017-1c264e513100 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -7.751384676473903, -2.9726830221804086, -2.0541437602045227, -1.0179313046149736
LBFGS Accumulation History: 3 points
Removed measurement 1322afc6 to history. Total: 3
th(0)=-7.751384676473903;dx=-9.831956605763838E-7
New Minimum: -7.751384676473903 > -8.729100250599737
WOLF (strong): th(2520000.0)=-8.729100250599737; dx=2.59538027091282E-7 evalInputDelta=0.9777155741258339
New Minimum: -8.729100250599737 > -9.255296535485394
END: th(1260000.0)=-9.255296535485394; dx=-6.046085650097352E-8 evalInputDelta=1.5039118590114908
Fitness changed from -7.751384676473903 to -9.255296535485394
Iteration 9 complete. Error: -9.255296535485394 Total: 0.5934; Orientation: 0.0444; Line Search: 0.4083
Adding measurement 4b76a1e8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.088e+04, gradient 7.366e-04, dot -0.604; [5a65d381-045d-40f1-8017-1c264e513100 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -9.255296535485394, -2.9726830221804086, -2.0541437602045227, -1.0179313046149736
LBFGS Accumulation History: 3 points
Removed measurement 4b76a1e8 to history. Total: 3
th(0)=-9.255296535485394;dx=-5.392954716747889E-7
New Minimum: -9.255296535485394 > -10.274555694097682
WOLF (strong): th(2714587.7094401736)=-10.274555694097682; dx=5.3742955580699824E-8 evalInputDelta=1.0192591586122877
END: th(1357293.8547200868)=-10.240670318251594; dx=-1.1882370369815046E-7 evalInputDelta=0.9853737827661995
Fitness changed from -9.255296535485394 to -10.274555694097682
Iteration 10 complete. Error: -10.274555694097682 Total: 0.5979; Orientation: 0.0443; Line Search: 0.4175
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 5003ffd1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.974e+04, gradient 9.741e-04, dot -0.420; [5a65d381-045d-40f1-8017-1c264e513100 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.274555694097682, -2.9726830221804086, -2.0541437602045227, -1.0179313046149736
LBFGS Accumulation History: 3 points
Removed measurement 5003ffd1 to history. Total: 3
th(0)=-10.274555694097682;dx=-9.400730002369981E-7
New Minimum: -10.274555694097682 > -11.412897672075454
WOLF (strong): th(2924200.965176051)=-11.412897672075454; dx=2.764836296932572E-7 evalInputDelta=1.1383419779777721
New Minimum: -11.412897672075454 > -12.005079713984074
END: th(1462100.4825880255)=-12.005079713984074; dx=-6.337040996642708E-8 evalInputDelta=1.730524019886392
Fitness changed from -10.274555694097682 to -12.005079713984074
Iteration 11 complete. Error: -12.005079713984074 Total: 0.6074; Orientation: 0.0445; Line Search: 0.4151
Adding measurement c5b815a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.925e+04, gradient 6.837e-04, dot -0.606; [5a65d381-045d-40f1-8017-1c264e513100 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.005079713984074, -2.9726830221804086, -2.0541437602045227, -1.0179313046149736
LBFGS Accumulation History: 3 points
Removed measurement c5b815a to history. Total: 3
th(0)=-12.005079713984074;dx=-4.6063268247405915E-7
New Minimum: -12.005079713984074 > -13.204680527741901
WOLF (strong): th(3150000.0000000005)=-13.204680527741901; dx=2.3935603136602118E-8 evalInputDelta=1.1996008137578276
END: th(1575000.0000000002)=-13.070055843211431; dx=-1.3427898621436062E-7 evalInputDelta=1.0649761292273574
Fitness changed from -12.005079713984074 to -13.204680527741901
Iteration 12 complete. Error: -13.204680527741901 Total: 0.5991; Orientation: 0.0443; Line Search: 0.4208
Adding measurement 73b0f742 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.702e+04, gradient 9.085e-04, dot -0.423; [5a65d381-045d-40f1-8017-1c264e513100 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -13.204680527741901, -2.9726830221804086, -2.0541437602045227, -1.0179313046149736
LBFGS Accumulation History: 3 points
Removed measurement 73b0f742 to history. Total: 3
th(0)=-13.204680527741901;dx=-8.206154066070349E-7
New Minimum: -13.204680527741901 > -14.214684308585694
WOLF (strong): th(3393234.6368002174)=-14.214684308585694; dx=2.542562169722581E-7 evalInputDelta=1.0100037808437925
New Minimum: -14.214684308585694 > -14.85976439301086
END: th(1696617.3184001087)=-14.85976439301086; dx=-2.3503062258496038E-8 evalInputDelta=1.6550838652689581
Fitness changed from -13.204680527741901 to -14.85976439301086
Iteration 13 complete. Error: -14.85976439301086 Total: 0.5988; Orientation: 0.0425; Line Search: 0.4175
Adding measurement 5f810e87 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.135e+04, gradient 6.540e-04, dot -0.570; [5a65d381-045d-40f1-8017-1c264e513100 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -14.85976439301086, -2.9726830221804086, -2.0541437602045227, -1.0179313046149736
LBFGS Accumulation History: 3 points
Removed measurement 5f810e87 to history. Total: 3
th(0)=-14.85976439301086;dx=-4.2178371741744493E-7
New Minimum: -14.85976439301086 > -15.902855815230055
WOLF (strong): th(3655251.2064700644)=-15.902855815230055; dx=7.388698198467827E-8 evalInputDelta=1.0430914222191952
END: th(1827625.6032350322)=-15.902412653367952; dx=-9.36599531079777E-8 evalInputDelta=1.0426482603570921
Fitness changed from -14.85976439301086 to -15.902855815230055
Iteration 14 complete. Error: -15.902855815230055 Total: 0.5918; Orientation: 0.0430; Line Search: 0.4123
Adding measurement 33f5f7b0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.374e+04, gradient 8.897e-04, dot -0.373; [5a65d381-045d-40f1-8017-1c264e513100 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -15.902855815230055, -2.9726830221804086, -2.0541437602045227, -1.0179313046149736
LBFGS Accumulation History: 3 points
Removed measurement 33f5f7b0 to history. Total: 3
th(0)=-15.902855815230055;dx=-7.874908392462888E-7
New Minimum: -15.902855815230055 > -16.45642159459479
WOLF (strong): th(3937500.0000000014)=-16.45642159459479; dx=3.181591202241473E-7 evalInputDelta=0.5535657793647353
New Minimum: -16.45642159459479 > -17.542386398372134
WOLF (strong): th(1968750.0000000007)=-17.542386398372134; dx=4.9763671355887873E-8 evalInputDelta=1.6395305831420792
END: th(656250.0000000002)=-16.891478834151002; dx=-4.958480626032834E-7 evalInputDelta=0.9886230189209471
Fitness changed from -15.902855815230055 to -17.542386398372134
Iteration 15 complete. Error: -17.542386398372134 Total: 1.2138; Orientation: 0.0433; Line Search: 1.0272
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -17.542386398372134 (> -Infinity) after 11.000s (< 5400.000s)
Returns
-17.542386398372134