BasicOptimizer.scala:89 executed in 60.26 seconds (0.794 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 165831253542200
Reset training subject: 165832123976500
Adding measurement 32cac372 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.7200717193033083;dx=-2.6375285205981E-4
New Minimum: -0.7200717193033083 > -0.720630831119265
WOLFE (weak): th(2.154434690031884)=-0.720630831119265; dx=-2.672709742647468E-4 evalInputDelta=5.591118159566921E-4
New Minimum: -0.720630831119265 > -0.7211904047202499
WOLFE (weak): th(4.308869380063768)=-0.7211904047202499; dx=-2.657552101436501E-4 evalInputDelta=0.0011186854169416005
New Minimum: -0.7211904047202499 > -0.7234322330675174
WOLFE (weak): th(12.926608140191302)=-0.7234322330675174; dx=-2.653618661515892E-4 evalInputDelta=0.0033605137642090943
New Minimum: -0.7234322330675174 > -0.7335582066967196
WOLFE (weak): th(51.70643256076521)=-0.7335582066967196; dx=-2.677951086374194E-4 evalInputDelta=0.013486487393411295
New Minimum: -0.7335582066967196 > -0.7888653206042422
WOLFE (weak): th(258.53216280382605)=-0.7888653206042422; dx=-2.7346033930235986E-4 evalInputDelta=0.06879360130093393
New Minimum: -0.7888653206042422 > -1.1397187350556275
WOLFE (weak): th(1551.1929768229563)=-1.1397187350556275; dx=-2.6190936614877596E-4 evalInputDelta=0.4196470157523192
New Minimum: -1.1397187350556275 > -1.8103972174401755
WOLF (strong): th(10858.350837760694)=-1.8103972174401755; dx=1.7839654369221972E-5 evalInputDelta=1.0903254981368673
New Minimum: -1.8103972174401755 > -1.9760830503595224
END: th(6204.771907291825)=-1.9760830503595224; dx=-8.971744397111459E-5 evalInputDelta=1.256011331056214
Fitness changed from -0.7200717193033083 to -1.9760830503595224
Iteration 1 complete. Error: -1.9760830503595224 Total: 10.2242; Orientation: 0.0145; Line Search: 7.7086
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement ee7eacf to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-1.9760830503595224;dx=-3.8881318420583397E-4
New Minimum: -1.9760830503595224 > -2.278359154970321
END: th(13367.775840804805)=-2.278359154970321; dx=-1.0563754882087192E-5 evalInputDelta=0.3022761046107987
Fitness changed from -1.9760830503595224 to -2.278359154970321
Iteration 2 complete. Error: -2.278359154970321 Total: 2.6187; Orientation: 0.0147; Line Search: 1.6954
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 65bd3b86 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-2.278359154970321;dx=-3.8966430280261226E-4
Armijo: th(28800.000000000004)=-1.6242533478734857; dx=4.1718672812004145E-5 evalInputDelta=-0.6541058070968353
New Minimum: -2.278359154970321 > -3.5042561946465964
WOLF (strong): th(14400.000000000002)=-3.5042561946465964; dx=1.8838716275586845E-5 evalInputDelta=1.2258970396762754
New Minimum: -3.5042561946465964 > -3.889221928733077
END: th(4800.000000000001)=-3.889221928733077; dx=-1.805813546118729E-4 evalInputDelta=1.610862773762756
Fitness changed from -2.278359154970321 to -3.889221928733077
Iteration 3 complete. Error: -3.889221928733077 Total: 4.2522; Orientation: 0.0151; Line Search: 3.3047
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 21d47ca0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.939e+02, gradient 1.749e-02, dot -0.401; [b427cc2f-8da1-4768-99c9-7b7de2dc820e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.889221928733077, -2.278359154970321, -1.9760830503595224, -0.7200717193033083
LBFGS Accumulation History: 3 points
Removed measurement 21d47ca0 to history. Total: 3
th(0)=-3.889221928733077;dx=-3.034712228023219E-4
New Minimum: -3.889221928733077 > -5.2270162238915185
END: th(10341.286512153045)=-5.2270162238915185; dx=-3.611090771694071E-5 evalInputDelta=1.3377942951584414
Fitness changed from -3.889221928733077 to -5.2270162238915185
Iteration 4 complete. Error: -5.2270162238915185 Total: 3.0442; Orientation: 0.2512; Line Search: 1.8862
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 7664a73d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.813e+02, gradient 2.073e-02, dot -0.267; [b427cc2f-8da1-4768-99c9-7b7de2dc820e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.2270162238915185, -2.278359154970321, -1.9760830503595224, -0.7200717193033083
LBFGS Accumulation History: 3 points
Removed measurement 7664a73d to history. Total: 3
th(0)=-5.2270162238915185;dx=-4.275216968807049E-4
Armijo: th(22279.626401341346)=-4.369561673117436; dx=3.128394834171945E-5 evalInputDelta=-0.857454550774083
New Minimum: -5.2270162238915185 > -6.538359338266197
WOLF (strong): th(11139.813200670673)=-6.538359338266197; dx=5.2034299706797966E-6 evalInputDelta=1.3113431143746785
New Minimum: -6.538359338266197 > -6.703819765968664
END: th(3713.271066890224)=-6.703819765968664; dx=-2.251857753576013E-4 evalInputDelta=1.4768035420771453
Fitness changed from -5.2270162238915185 to -6.703819765968664
Iteration 5 complete. Error: -6.703819765968664 Total: 4.4269; Orientation: 0.2616; Line Search: 3.2992
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 548d0563 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.787e+02, gradient 1.878e-02, dot -0.316; [b427cc2f-8da1-4768-99c9-7b7de2dc820e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.703819765968664, -2.278359154970321, -1.9760830503595224, -0.7200717193033083
LBFGS Accumulation History: 3 points
Removed measurement 548d0563 to history. Total: 3
th(0)=-6.703819765968664;dx=-3.514232677561817E-4
New Minimum: -6.703819765968664 > -8.203211598890839
END: th(8000.000000000003)=-8.203211598890839; dx=-1.0651616915116013E-4 evalInputDelta=1.499391832922175
Fitness changed from -6.703819765968664 to -8.203211598890839
Iteration 6 complete. Error: -8.203211598890839 Total: 2.8720; Orientation: 0.2545; Line Search: 1.7149
Adding measurement 771d28ab to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.511e+03, gradient 2.184e-02, dot -0.297; [b427cc2f-8da1-4768-99c9-7b7de2dc820e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.203211598890839, -2.278359154970321, -1.9760830503595224, -0.7200717193033083
LBFGS Accumulation History: 3 points
Removed measurement 771d28ab to history. Total: 3
th(0)=-8.203211598890839;dx=-4.732206089168499E-4
Armijo: th(17235.477520255077)=-6.823722082911032; dx=6.439264133828654E-5 evalInputDelta=-1.3794895159798068
New Minimum: -8.203211598890839 > -9.086723283766036
WOLF (strong): th(8617.738760127539)=-9.086723283766036; dx=3.431045459431467E-5 evalInputDelta=0.8835116848751969
New Minimum: -9.086723283766036 > -9.295714930985788
END: th(2872.5795867091797)=-9.295714930985788; dx=-2.6239344922904224E-4 evalInputDelta=1.0925033320949495
Fitness changed from -8.203211598890839 to -9.295714930985788
Iteration 7 complete. Error: -9.295714930985788 Total: 4.6150; Orientation: 0.3705; Line Search: 3.4662
Adding measurement 64f64076 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.768e+03, gradient 1.935e-02, dot -0.328; [b427cc2f-8da1-4768-99c9-7b7de2dc820e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -9.295714930985788, -2.278359154970321, -1.9760830503595224, -0.7200717193033083
LBFGS Accumulation History: 3 points
Removed measurement 64f64076 to history. Total: 3
th(0)=-9.295714930985788;dx=-3.710908470148601E-4
New Minimum: -9.295714930985788 > -10.34698104608423
END: th(6188.7851114837085)=-10.34698104608423; dx=-1.2561619307077513E-4 evalInputDelta=1.051266115098441
Fitness changed from -9.295714930985788 to -10.34698104608423
Iteration 8 complete. Error: -10.34698104608423 Total: 2.7662; Orientation: 0.2533; Line Search: 1.7100
Adding measurement 18bcb997 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.536e+03, gradient 2.367e-02, dot -0.271; [b427cc2f-8da1-4768-99c9-7b7de2dc820e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.34698104608423, -2.278359154970321, -1.9760830503595224, -0.7200717193033083
LBFGS Accumulation History: 3 points
Removed measurement 18bcb997 to history. Total: 3
th(0)=-10.34698104608423;dx=-5.435984394410638E-4
Armijo: th(13333.333333333341)=-8.584563026858746; dx=9.817434371297578E-5 evalInputDelta=-1.7624180192254837
New Minimum: -10.34698104608423 > -10.625532200175149
WOLF (strong): th(6666.666666666671)=-10.625532200175149; dx=8.649504123733369E-5 evalInputDelta=0.2785511540909198
New Minimum: -10.625532200175149 > -11.282744550426127
END: th(2222.2222222222235)=-11.282744550426127; dx=-2.4183668471713447E-4 evalInputDelta=0.9357635043418977
Fitness changed from -10.34698104608423 to -11.282744550426127
Iteration 9 complete. Error: -11.282744550426127 Total: 4.5740; Orientation: 0.2646; Line Search: 3.5204
Adding measurement 3eb75ce5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.486e+03, gradient 1.859e-02, dot -0.352; [b427cc2f-8da1-4768-99c9-7b7de2dc820e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.282744550426127, -2.278359154970321, -1.9760830503595224, -0.7200717193033083
LBFGS Accumulation History: 3 points
Removed measurement 3eb75ce5 to history. Total: 3
th(0)=-11.282744550426127;dx=-3.4072619881222554E-4
New Minimum: -11.282744550426127 > -12.09460718371806
END: th(4787.6326445153)=-12.09460718371806; dx=-1.6016546436445662E-4 evalInputDelta=0.8118626332919323
Fitness changed from -11.282744550426127 to -12.09460718371806
Iteration 10 complete. Error: -12.09460718371806 Total: 2.6651; Orientation: 0.2498; Line Search: 1.6274
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 1c9781ca to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.321e+03, gradient 2.176e-02, dot -0.286; [b427cc2f-8da1-4768-99c9-7b7de2dc820e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.09460718371806, -2.278359154970321, -1.9760830503595224, -0.7200717193033083
LBFGS Accumulation History: 3 points
Removed measurement 1c9781ca to history. Total: 3
th(0)=-12.09460718371806;dx=-4.6966044694719E-4
Armijo: th(10314.641852472849)=-11.814101257237473; dx=7.207671748376639E-6 evalInputDelta=-0.28050592648058625
New Minimum: -12.09460718371806 > -12.572379754362604
END: th(5157.320926236424)=-12.572379754362604; dx=-5.362335215267041E-6 evalInputDelta=0.4777725706445448
Fitness changed from -12.09460718371806 to -12.572379754362604
Iteration 11 complete. Error: -12.572379754362604 Total: 3.6170; Orientation: 0.2564; Line Search: 2.4612
Adding measurement 4e189df7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.151e+03, gradient 2.510e-02, dot -0.251; [b427cc2f-8da1-4768-99c9-7b7de2dc820e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.572379754362604, -2.278359154970321, -1.9760830503595224, -0.7200717193033083
LBFGS Accumulation History: 3 points
Removed measurement 4e189df7 to history. Total: 3
th(0)=-12.572379754362604;dx=-6.2707640446248E-4
Armijo: th(11111.111111111119)=-11.408416648455377; dx=6.736108667637144E-5 evalInputDelta=-1.1639631059072268
New Minimum: -12.572379754362604 > -13.169958781879712
WOLF (strong): th(5555.555555555559)=-13.169958781879712; dx=1.3450531808460798E-4 evalInputDelta=0.5975790275171082
New Minimum: -13.169958781879712 > -13.632658441951596
END: th(1851.851851851853)=-13.632658441951596; dx=-4.005879975719984E-4 evalInputDelta=1.0602786875889922
Fitness changed from -12.572379754362604 to -13.632658441951596
Iteration 12 complete. Error: -13.632658441951596 Total: 4.5460; Orientation: 0.2530; Line Search: 3.5136
Adding measurement 4fcdebe8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.512e+04, gradient 1.976e-02, dot -0.346; [b427cc2f-8da1-4768-99c9-7b7de2dc820e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -13.632658441951596, -2.278359154970321, -1.9760830503595224, -0.7200717193033083
LBFGS Accumulation History: 3 points
Removed measurement 4fcdebe8 to history. Total: 3
th(0)=-13.632658441951596;dx=-3.876303512578554E-4
New Minimum: -13.632658441951596 > -14.224400873528154
END: th(3989.693870429417)=-14.224400873528154; dx=-9.69209913354859E-5 evalInputDelta=0.5917424315765576
Fitness changed from -13.632658441951596 to -14.224400873528154
Iteration 13 complete. Error: -14.224400873528154 Total: 2.7182; Orientation: 0.2522; Line Search: 1.6805
Adding measurement 29d484cc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.570e+04, gradient 2.389e-02, dot -0.262; [b427cc2f-8da1-4768-99c9-7b7de2dc820e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -14.224400873528154, -2.278359154970321, -1.9760830503595224, -0.7200717193033083
LBFGS Accumulation History: 3 points
Removed measurement 29d484cc to history. Total: 3
th(0)=-14.224400873528154;dx=-5.6767741989439E-4
New Minimum: -14.224400873528154 > -14.226070147087677
END: th(8595.534877060709)=-14.226070147087677; dx=-4.702363510949432E-5 evalInputDelta=0.0016692735595231767
Fitness changed from -14.224400873528154 to -14.226070147087677
Iteration 14 complete. Error: -14.226070147087677 Total: 2.7513; Orientation: 0.2657; Line Search: 1.7007
Adding measurement 3095858 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.787e+04, gradient 2.335e-02, dot -0.254; [b427cc2f-8da1-4768-99c9-7b7de2dc820e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -14.226070147087677, -2.278359154970321, -1.9760830503595224, -0.7200717193033083
LBFGS Accumulation History: 3 points
Removed measurement 3095858 to history. Total: 3
th(0)=-14.226070147087677;dx=-5.410715246792825E-4
Armijo: th(18518.518518518536)=-11.23330829156292; dx=1.1510887468199214E-4 evalInputDelta=-2.9927618555247566
New Minimum: -14.226070147087677 > -15.063682508358152
WOLF (strong): th(9259.259259259268)=-15.063682508358152; dx=2.9990403859835787E-5 evalInputDelta=0.8376123612704749
New Minimum: -15.063682508358152 > -15.57581815180456
END: th(3086.4197530864226)=-15.57581815180456; dx=-2.990790841663934E-4 evalInputDelta=1.3497480047168828
Fitness changed from -14.226070147087677 to -15.57581815180456
Iteration 15 complete. Error: -15.57581815180456 Total: 4.4642; Orientation: 0.2511; Line Search: 3.4482
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -15.57581815180456 (> -Infinity) after 60.262s (< 5400.000s)
Returns
-15.57581815180456