BasicOptimizer.scala:89 executed in 9.14 seconds (0.566 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 140150326505100
Reset training subject: 140150456665800
Adding measurement 3eb984fa to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.20338105077508178;dx=-9.707121118988866E-6
New Minimum: -0.20338105077508178 > -0.2033956840206455
WOLFE (weak): th(2.154434690031884)=-0.2033956840206455; dx=-9.68410620132398E-6 evalInputDelta=1.4633245563722141E-5
New Minimum: -0.2033956840206455 > -0.20341019944038943
WOLFE (weak): th(4.308869380063768)=-0.20341019944038943; dx=-9.60273243967619E-6 evalInputDelta=2.9148665307648924E-5
New Minimum: -0.20341019944038943 > -0.20346816904617288
WOLFE (weak): th(12.926608140191302)=-0.20346816904617288; dx=-9.739570682702134E-6 evalInputDelta=8.711827109109116E-5
New Minimum: -0.20346816904617288 > -0.2037301281286799
WOLFE (weak): th(51.70643256076521)=-0.2037301281286799; dx=-9.633966800529182E-6 evalInputDelta=3.490773535981262E-4
New Minimum: -0.2037301281286799 > -0.20513989247195355
WOLFE (weak): th(258.53216280382605)=-0.20513989247195355; dx=-9.867095355446157E-6 evalInputDelta=0.0017588416968717657
New Minimum: -0.20513989247195355 > -0.21421004382762557
WOLFE (weak): th(1551.1929768229563)=-0.21421004382762557; dx=-1.0059744907925998E-5 evalInputDelta=0.010828993052543784
New Minimum: -0.21421004382762557 > -0.29217731896383753
WOLFE (weak): th(10858.350837760694)=-0.29217731896383753; dx=-9.149195514483728E-6 evalInputDelta=0.08879626818875574
New Minimum: -0.29217731896383753 > -0.6284021667210737
END: th(86866.80670208555)=-0.6284021667210737; dx=-1.2106622658422243E-6 evalInputDelta=0.42502111594599196
Fitness changed from -0.20338105077508178 to -0.6284021667210737
Iteration 1 complete. Error: -0.6284021667210737 Total: 1.4418; Orientation: 0.0016; Line Search: 1.0900
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 50211d51 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-0.6284021667210737;dx=-2.6967456305144854E-5
New Minimum: -0.6284021667210737 > -0.9850769921827176
END: th(187148.86177126726)=-0.9850769921827176; dx=-7.715163433163431E-7 evalInputDelta=0.35667482546164386
Fitness changed from -0.6284021667210737 to -0.9850769921827176
Iteration 2 complete. Error: -0.9850769921827176 Total: 0.3652; Orientation: 0.0016; Line Search: 0.2383
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 36053a00 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-0.9850769921827176;dx=-9.23889656348322E-6
New Minimum: -0.9850769921827176 > -2.079994429128585
END: th(403200.00000000006)=-2.079994429128585; dx=-1.3218925295414075E-7 evalInputDelta=1.0949174369458672
Fitness changed from -0.9850769921827176 to -2.079994429128585
Iteration 3 complete. Error: -2.079994429128585 Total: 0.3545; Orientation: 0.0014; Line Search: 0.2286
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 3c61a3e9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.869e+02, gradient 3.474e-03, dot -0.211; [89992269-4496-4628-835e-6b1bb01e326b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.079994429128585, -0.9850769921827176, -0.6284021667210737, -0.20338105077508178
LBFGS Accumulation History: 3 points
Removed measurement 3c61a3e9 to history. Total: 3
th(0)=-2.079994429128585;dx=-1.1904427673740152E-5
New Minimum: -2.079994429128585 > -3.0071662582258765
END: th(868668.0670208557)=-3.0071662582258765; dx=-3.553588836835365E-7 evalInputDelta=0.9271718290972917
Fitness changed from -2.079994429128585 to -3.0071662582258765
Iteration 4 complete. Error: -3.0071662582258765 Total: 0.3811; Orientation: 0.0281; Line Search: 0.2308
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement a732338 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.131e+03, gradient 2.576e-03, dot -0.199; [89992269-4496-4628-835e-6b1bb01e326b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.0071662582258765, -0.9850769921827176, -0.6284021667210737, -0.20338105077508178
LBFGS Accumulation History: 3 points
Removed measurement a732338 to history. Total: 3
th(0)=-3.0071662582258765;dx=-6.434084875804491E-6
New Minimum: -3.0071662582258765 > -3.78463591902374
END: th(1871488.617712673)=-3.78463591902374; dx=-2.0933967990698636E-9 evalInputDelta=0.7774696607978635
Fitness changed from -3.0071662582258765 to -3.78463591902374
Iteration 5 complete. Error: -3.78463591902374 Total: 0.3837; Orientation: 0.0324; Line Search: 0.2312
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 2d370c58 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.522e+03, gradient 2.371e-03, dot -0.177; [89992269-4496-4628-835e-6b1bb01e326b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.78463591902374, -0.9850769921827176, -0.6284021667210737, -0.20338105077508178
LBFGS Accumulation History: 3 points
Removed measurement 2d370c58 to history. Total: 3
th(0)=-3.78463591902374;dx=-5.500747066171552E-6
New Minimum: -3.78463591902374 > -6.08999555565425
WOLF (strong): th(4032000.0000000014)=-6.08999555565425; dx=4.253322779033973E-7 evalInputDelta=2.30535963663051
New Minimum: -6.08999555565425 > -8.527917714370915
WOLF (strong): th(2016000.0000000007)=-8.527917714370915; dx=1.0354309973787375E-7 evalInputDelta=4.743281795347174
END: th(672000.0000000002)=-7.731606700043433; dx=-2.249908223364194E-6 evalInputDelta=3.9469707810196932
Fitness changed from -3.78463591902374 to -8.527917714370915
Iteration 6 complete. Error: -8.527917714370915 Total: 0.6060; Orientation: 0.0278; Line Search: 0.4543
Adding measurement 74b63906 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.984e+03, gradient 2.589e-03, dot -0.217; [89992269-4496-4628-835e-6b1bb01e326b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.527917714370915, -0.9850769921827176, -0.6284021667210737, -0.20338105077508178
LBFGS Accumulation History: 3 points
Removed measurement 74b63906 to history. Total: 3
th(0)=-8.527917714370915;dx=-6.522729146830222E-6
New Minimum: -8.527917714370915 > -11.535379370601605
END: th(1447780.1117014263)=-11.535379370601605; dx=-5.428707536754916E-7 evalInputDelta=3.0074616562306904
Fitness changed from -8.527917714370915 to -11.535379370601605
Iteration 7 complete. Error: -11.535379370601605 Total: 0.3772; Orientation: 0.0289; Line Search: 0.2366
Adding measurement af025dc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.079e+04, gradient 3.184e-03, dot -0.215; [89992269-4496-4628-835e-6b1bb01e326b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.535379370601605, -0.9850769921827176, -0.6284021667210737, -0.20338105077508178
LBFGS Accumulation History: 3 points
Removed measurement af025dc to history. Total: 3
th(0)=-11.535379370601605;dx=-9.888504615920243E-6
Armijo: th(3119147.6961877886)=-9.39722733555997; dx=8.786215140498555E-7 evalInputDelta=-2.138152035041635
New Minimum: -11.535379370601605 > -11.88020378254356
WOLF (strong): th(1559573.8480938943)=-11.88020378254356; dx=4.992270242157783E-7 evalInputDelta=0.34482441194195523
New Minimum: -11.88020378254356 > -12.762282874530065
END: th(519857.94936463144)=-12.762282874530065; dx=-9.621027621330563E-7 evalInputDelta=1.22690350392846
Fitness changed from -11.535379370601605 to -12.762282874530065
Iteration 8 complete. Error: -12.762282874530065 Total: 0.6016; Orientation: 0.0287; Line Search: 0.4583
Adding measurement 35ed04df to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.282e+04, gradient 2.580e-03, dot -0.280; [89992269-4496-4628-835e-6b1bb01e326b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.762282874530065, -0.9850769921827176, -0.6284021667210737, -0.20338105077508178
LBFGS Accumulation History: 3 points
Removed measurement 35ed04df to history. Total: 3
th(0)=-12.762282874530065;dx=-6.430458491436092E-6
New Minimum: -12.762282874530065 > -15.078389299373887
END: th(1120000.0000000005)=-15.078389299373887; dx=-6.10299967143878E-7 evalInputDelta=2.316106424843822
Fitness changed from -12.762282874530065 to -15.078389299373887
Iteration 9 complete. Error: -15.078389299373887 Total: 0.3703; Orientation: 0.0286; Line Search: 0.2331
Adding measurement 6545c3a9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.356e+05, gradient 2.824e-03, dot -0.290; [89992269-4496-4628-835e-6b1bb01e326b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -15.078389299373887, -0.9850769921827176, -0.6284021667210737, -0.20338105077508178
LBFGS Accumulation History: 3 points
Removed measurement 6545c3a9 to history. Total: 3
th(0)=-15.078389299373887;dx=-7.780374439952582E-6
Armijo: th(2412966.852835711)=-11.596011190511215; dx=1.258389877857206E-6 evalInputDelta=-3.4823781088626724
Armijo: th(1206483.4264178555)=-14.580652534423871; dx=2.81486556018035E-7 evalInputDelta=-0.49773676495001595
New Minimum: -15.078389299373887 > -16.091951391253797
END: th(402161.14213928516)=-16.091951391253797; dx=-9.618634114763466E-7 evalInputDelta=1.0135620918799102
Fitness changed from -15.078389299373887 to -16.091951391253797
Iteration 10 complete. Error: -16.091951391253797 Total: 0.6125; Orientation: 0.0278; Line Search: 0.4707
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 6bdf0780 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.057e+04, gradient 2.769e-03, dot 0.435; [89992269-4496-4628-835e-6b1bb01e326b = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-16.091951391253797;dx=-12.531510083478697
Armijo: th(866429.9156077192)=-10.258621120757091; dx=4.572738437402608E-6 evalInputDelta=-5.833330270496706
Armijo: th(433214.9578038596)=-10.258621120757091; dx=9.232786246038669E-6 evalInputDelta=-5.833330270496706
Armijo: th(144404.98593461988)=-10.258621120757091; dx=2.7537311123170356E-5 evalInputDelta=-5.833330270496706
Armijo: th(36101.24648365497)=-10.258621120757091; dx=1.1166004969835087E-4 evalInputDelta=-5.833330270496706
Armijo: th(7220.249296730994)=-10.266771186342396; dx=5.513034327029366E-4 evalInputDelta=-5.825180204911401
Armijo: th(1203.374882788499)=-10.30600088122915; dx=0.003266634297972362 evalInputDelta=-5.785950510024648
Armijo: th(171.91069754121415)=-10.361669098635184; dx=0.023047453746395222 evalInputDelta=-5.730282292618613
Armijo: th(21.48883719265177)=-10.635601593657597; dx=0.1816722811781701 evalInputDelta=-5.4563497975962
Armijo: th(2.3876485769613076)=-13.829705947705445; dx=1.1501942892041004 evalInputDelta=-2.262245443548352
New Minimum: -16.091951391253797 > -16.95638127699779
END: th(0.23876485769613076)=-16.95638127699779; dx=-3.324523388004379 evalInputDelta=0.8644298857439914
Fitness changed from -16.091951391253797 to -16.95638127699779
Iteration 11 complete. Error: -16.95638127699779 Total: 2.0867; Orientation: 0.0307; Line Search: 1.9270
Adding measurement 48964f3d to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.065e+06, gradient 2.432e-03, dot -0.310; [89992269-4496-4628-835e-6b1bb01e326b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.95638127699779, -16.091951391253797, -0.9850769921827176, -0.6284021667210737, -0.20338105077508178
Rejected: LBFGS Orientation magnitude: 2.087e+06, gradient 2.432e-03, dot -0.320; [89992269-4496-4628-835e-6b1bb01e326b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.95638127699779, -16.091951391253797, -0.9850769921827176, -0.6284021667210737
LBFGS Accumulation History: 3 points
Removed measurement 48964f3d to history. Total: 4
Removed measurement 6bdf0780 to history. Total: 3
th(0)=-16.95638127699779;dx=-5.650457156889566E-6
New Minimum: -16.95638127699779 > -16.95638299832608
END: th(0.5144032921810703)=-16.95638299832608; dx=-3.563604167013756E-6 evalInputDelta=1.7213282923478346E-6
Fitness changed from -16.95638127699779 to -16.95638299832608
Iteration 12 complete. Error: -16.95638299832608 Total: 0.4203; Orientation: 0.0673; Line Search: 0.2428
Adding measurement 6e8fd272 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.806e+06, gradient 2.425e-03, dot -0.315; [89992269-4496-4628-835e-6b1bb01e326b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.95638299832608, -0.9850769921827176, -0.6284021667210737, -0.20338105077508178
LBFGS Accumulation History: 3 points
Removed measurement 6e8fd272 to history. Total: 3
th(0)=-16.95638299832608;dx=-5.702391082860455E-6
New Minimum: -16.95638299832608 > -16.95638414587818
END: th(1.1082482973415047)=-16.95638414587818; dx=-3.631898440330202E-6 evalInputDelta=1.1475520977910492E-6
Fitness changed from -16.95638299832608 to -16.95638414587818
Iteration 13 complete. Error: -16.95638414587818 Total: 0.3773; Orientation: 0.0298; Line Search: 0.2322
Adding measurement 716ea64 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.399e+06, gradient 2.427e-03, dot -0.311; [89992269-4496-4628-835e-6b1bb01e326b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.95638414587818, -0.9850769921827176, -0.6284021667210737, -0.20338105077508178
LBFGS Accumulation History: 3 points
Removed measurement 716ea64 to history. Total: 3
th(0)=-16.95638414587818;dx=-5.734465771099222E-6
New Minimum: -16.95638414587818 > -16.956393900067877
END: th(2.3876485769613076)=-16.956393900067877; dx=-3.5936937964146857E-6 evalInputDelta=9.754189697730453E-6
Fitness changed from -16.95638414587818 to -16.956393900067877
Iteration 14 complete. Error: -16.956393900067877 Total: 0.3774; Orientation: 0.0290; Line Search: 0.2351
Adding measurement 20501184 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.192e+07, gradient 2.424e-03, dot -0.312; [89992269-4496-4628-835e-6b1bb01e326b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.956393900067877, -0.9850769921827176, -0.6284021667210737, -0.20338105077508178
LBFGS Accumulation History: 3 points
Removed measurement 20501184 to history. Total: 3
th(0)=-16.956393900067877;dx=-5.789486065755787E-6
New Minimum: -16.956393900067877 > -16.95640996577985
END: th(5.1440329218107035)=-16.95640996577985; dx=-3.5928686480728462E-6 evalInputDelta=1.6065711974988517E-5
Fitness changed from -16.956393900067877 to -16.95640996577985
Iteration 15 complete. Error: -16.95640996577985 Total: 0.3721; Orientation: 0.0300; Line Search: 0.2315
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -16.95640996577985 (> -Infinity) after 9.139s (< 5400.000s)
Returns
-16.95640996577985