BasicOptimizer.scala:89 executed in 8.39 seconds (0.383 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 140910960519800
Reset training subject: 140911099495300
Adding measurement 1191af4f to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.7262943421228756;dx=-1.7264592300856967E-4
New Minimum: -0.7262943421228756 > -0.7267359875970631
WOLFE (weak): th(2.154434690031884)=-0.7267359875970631; dx=-1.7046716074640452E-4 evalInputDelta=4.416454741875686E-4
New Minimum: -0.7267359875970631 > -0.7271781022949786
WOLFE (weak): th(4.308869380063768)=-0.7271781022949786; dx=-1.6947006259674687E-4 evalInputDelta=8.837601721030097E-4
New Minimum: -0.7271781022949786 > -0.7289481403284712
WOLFE (weak): th(12.926608140191302)=-0.7289481403284712; dx=-1.7556157574552563E-4 evalInputDelta=0.0026537982055956366
New Minimum: -0.7289481403284712 > -0.7369313315827419
WOLFE (weak): th(51.70643256076521)=-0.7369313315827419; dx=-1.8218824889914813E-4 evalInputDelta=0.010636989459866353
New Minimum: -0.7369313315827419 > -0.7804129491517046
WOLFE (weak): th(258.53216280382605)=-0.7804129491517046; dx=-1.7382811289310622E-4 evalInputDelta=0.05411860702882898
New Minimum: -0.7804129491517046 > -1.0687327325257798
WOLFE (weak): th(1551.1929768229563)=-1.0687327325257798; dx=-2.102583194893464E-4 evalInputDelta=0.34243839040290425
New Minimum: -1.0687327325257798 > -2.583180455754794
END: th(10858.350837760694)=-2.583180455754794; dx=-8.376830410315386E-5 evalInputDelta=1.8568861136319184
Fitness changed from -0.7262943421228756 to -2.583180455754794
Iteration 1 complete. Error: -2.583180455754794 Total: 1.2871; Orientation: 0.0014; Line Search: 0.9240
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 576e93bd to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-2.583180455754794;dx=-3.3048202789459896E-4
New Minimum: -2.583180455754794 > -5.089785887901949
END: th(23393.607721408407)=-5.089785887901949; dx=-1.1218494840302105E-5 evalInputDelta=2.5066054321471554
Fitness changed from -2.583180455754794 to -5.089785887901949
Iteration 2 complete. Error: -5.089785887901949 Total: 0.3609; Orientation: 0.0015; Line Search: 0.2307
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 492c02c8 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-5.089785887901949;dx=-2.7956629341419797E-4
Armijo: th(50400.00000000001)=-4.933010858208106; dx=8.574551266914105E-6 evalInputDelta=-0.1567750296938435
New Minimum: -5.089785887901949 > -6.706279282084648
WOLF (strong): th(25200.000000000004)=-6.706279282084648; dx=2.1320984117906875E-5 evalInputDelta=1.616493394182699
New Minimum: -6.706279282084648 > -7.106992230330539
END: th(8400.000000000002)=-7.106992230330539; dx=-1.36029584285283E-4 evalInputDelta=2.0172063424285893
Fitness changed from -5.089785887901949 to -7.106992230330539
Iteration 3 complete. Error: -7.106992230330539 Total: 0.5844; Orientation: 0.0015; Line Search: 0.4640
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement fd69382 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.071e+03, gradient 1.512e-02, dot -0.354; [2a5d3207-7741-4d6e-9241-975ca8d3ac4a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -7.106992230330539, -5.089785887901949, -2.583180455754794, -0.7262943421228756
LBFGS Accumulation History: 3 points
Removed measurement fd69382 to history. Total: 3
th(0)=-7.106992230330539;dx=-2.1874351640556992E-4
New Minimum: -7.106992230330539 > -8.740076374030938
END: th(18097.251396267828)=-8.740076374030938; dx=-2.7518368479090535E-5 evalInputDelta=1.6330841437003993
Fitness changed from -7.106992230330539 to -8.740076374030938
Iteration 4 complete. Error: -8.740076374030938 Total: 0.3766; Orientation: 0.0281; Line Search: 0.2301
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 70f7c52d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.172e+03, gradient 1.844e-02, dot -0.260; [2a5d3207-7741-4d6e-9241-975ca8d3ac4a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.740076374030938, -5.089785887901949, -2.583180455754794, -0.7262943421228756
LBFGS Accumulation History: 3 points
Removed measurement 70f7c52d to history. Total: 3
th(0)=-8.740076374030938;dx=-3.251158966179055E-4
Armijo: th(38989.346202347355)=-6.822299886040698; dx=3.082847656432106E-5 evalInputDelta=-1.9177764879902401
New Minimum: -8.740076374030938 > -9.275043444204844
WOLF (strong): th(19494.673101173677)=-9.275043444204844; dx=4.9207202348476054E-5 evalInputDelta=0.5349670701739058
New Minimum: -9.275043444204844 > -10.331646692780753
END: th(6498.224367057893)=-10.331646692780753; dx=-9.9926533854711E-5 evalInputDelta=1.591570318749815
Fitness changed from -8.740076374030938 to -10.331646692780753
Iteration 5 complete. Error: -10.331646692780753 Total: 0.6023; Orientation: 0.0302; Line Search: 0.4529
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 4f35eada to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.246e+03, gradient 1.483e-02, dot -0.411; [2a5d3207-7741-4d6e-9241-975ca8d3ac4a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.331646692780753, -5.089785887901949, -2.583180455754794, -0.7262943421228756
LBFGS Accumulation History: 3 points
Removed measurement 4f35eada to history. Total: 3
th(0)=-10.331646692780753;dx=-2.1137730165809486E-4
New Minimum: -10.331646692780753 > -11.64983241369606
END: th(14000.000000000005)=-11.64983241369606; dx=-1.7943918262480425E-5 evalInputDelta=1.3181857209153076
Fitness changed from -10.331646692780753 to -11.64983241369606
Iteration 6 complete. Error: -11.64983241369606 Total: 0.3793; Orientation: 0.0275; Line Search: 0.2304
Adding measurement 533581bc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.456e+03, gradient 2.214e-02, dot -0.249; [2a5d3207-7741-4d6e-9241-975ca8d3ac4a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.64983241369606, -5.089785887901949, -2.583180455754794, -0.7262943421228756
LBFGS Accumulation History: 3 points
Removed measurement 533581bc to history. Total: 3
th(0)=-11.64983241369606;dx=-4.836543466762058E-4
Armijo: th(30162.085660446384)=-7.543943250546586; dx=4.154394350981108E-5 evalInputDelta=-4.105889163149475
Armijo: th(15081.042830223192)=-10.409357477973305; dx=9.754011473212624E-5 evalInputDelta=-1.2404749357227551
New Minimum: -11.64983241369606 > -12.792189375439285
END: th(5027.014276741064)=-12.792189375439285; dx=-4.8363171837995965E-5 evalInputDelta=1.142356961743225
Fitness changed from -11.64983241369606 to -12.792189375439285
Iteration 7 complete. Error: -12.792189375439285 Total: 0.9994; Orientation: 0.0286; Line Search: 0.8620
Adding measurement 14715e5e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.171e+03, gradient 1.558e-02, dot -0.429; [2a5d3207-7741-4d6e-9241-975ca8d3ac4a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.792189375439285, -5.089785887901949, -2.583180455754794, -0.7262943421228756
LBFGS Accumulation History: 3 points
Removed measurement 14715e5e to history. Total: 3
th(0)=-12.792189375439285;dx=-2.3082653553381337E-4
New Minimum: -12.792189375439285 > -13.868823684814036
END: th(10830.37394509649)=-13.868823684814036; dx=-1.0287910062864235E-5 evalInputDelta=1.0766343093747501
Fitness changed from -12.792189375439285 to -13.868823684814036
Iteration 8 complete. Error: -13.868823684814036 Total: 0.3731; Orientation: 0.0302; Line Search: 0.2328
Adding measurement 49f61877 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.380e+03, gradient 2.149e-02, dot -0.302; [2a5d3207-7741-4d6e-9241-975ca8d3ac4a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -13.868823684814036, -5.089785887901949, -2.583180455754794, -0.7262943421228756
LBFGS Accumulation History: 3 points
Removed measurement 49f61877 to history. Total: 3
th(0)=-13.868823684814036;dx=-4.521257724111215E-4
Armijo: th(23333.333333333347)=-10.36622731136756; dx=7.915946110878017E-5 evalInputDelta=-3.502596373446476
Armijo: th(11666.666666666673)=-13.586245590228836; dx=1.566855499202484E-4 evalInputDelta=-0.2825780945851992
New Minimum: -13.868823684814036 > -15.124238136565461
END: th(3888.888888888891)=-15.124238136565461; dx=-8.642060749926932E-5 evalInputDelta=1.2554144517514256
Fitness changed from -13.868823684814036 to -15.124238136565461
Iteration 9 complete. Error: -15.124238136565461 Total: 0.6015; Orientation: 0.0274; Line Search: 0.4593
Adding measurement 199259ad to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.077e+04, gradient 1.448e-02, dot -0.497; [2a5d3207-7741-4d6e-9241-975ca8d3ac4a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -15.124238136565461, -5.089785887901949, -2.583180455754794, -0.7262943421228756
LBFGS Accumulation History: 3 points
Removed measurement 199259ad to history. Total: 3
th(0)=-15.124238136565461;dx=-2.0140588678225777E-4
New Minimum: -15.124238136565461 > -16.126252267899787
END: th(8378.357127901774)=-16.126252267899787; dx=-8.016961832760792E-5 evalInputDelta=1.0020141313343256
Fitness changed from -15.124238136565461 to -16.126252267899787
Iteration 10 complete. Error: -16.126252267899787 Total: 0.3675; Orientation: 0.0288; Line Search: 0.2262
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 7d46c184 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.984e+04, gradient 1.629e-02, dot -0.445; [2a5d3207-7741-4d6e-9241-975ca8d3ac4a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.126252267899787, -5.089785887901949, -2.583180455754794, -0.7262943421228756
LBFGS Accumulation History: 3 points
Removed measurement 7d46c184 to history. Total: 3
th(0)=-16.126252267899787;dx=-2.5730540717903934E-4
New Minimum: -16.126252267899787 > -16.392574621395852
WOLF (strong): th(18050.623241827485)=-16.392574621395852; dx=1.2413266333369942E-5 evalInputDelta=0.26632235349606503
New Minimum: -16.392574621395852 > -17.024419939238502
END: th(9025.311620913742)=-17.024419939238502; dx=-2.2790097970647745E-5 evalInputDelta=0.8981676713387152
Fitness changed from -16.126252267899787 to -17.024419939238502
Iteration 11 complete. Error: -17.024419939238502 Total: 0.5096; Orientation: 0.0287; Line Search: 0.3510
Adding measurement 3c7135bc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.444e+04, gradient 1.758e-02, dot -0.384; [2a5d3207-7741-4d6e-9241-975ca8d3ac4a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.024419939238502, -5.089785887901949, -2.583180455754794, -0.7262943421228756
LBFGS Accumulation History: 3 points
Removed measurement 3c7135bc to history. Total: 3
th(0)=-17.024419939238502;dx=-2.970147463580509E-4
Armijo: th(19444.444444444456)=-16.711610376334644; dx=5.640430313694244E-5 evalInputDelta=-0.3128095629038583
New Minimum: -17.024419939238502 > -18.26286524437022
WOLF (strong): th(9722.222222222228)=-18.26286524437022; dx=1.6887074434938674E-5 evalInputDelta=1.2384453051317195
END: th(3240.7407407407427)=-18.04362239440313; dx=-1.6327689771590808E-4 evalInputDelta=1.0192024551646277
Fitness changed from -17.024419939238502 to -18.26286524437022
Iteration 12 complete. Error: -18.26286524437022 Total: 0.5996; Orientation: 0.0286; Line Search: 0.4589
Adding measurement 4ef1d79 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.509e+04, gradient 1.808e-02, dot -0.400; [2a5d3207-7741-4d6e-9241-975ca8d3ac4a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.26286524437022, -5.089785887901949, -2.583180455754794, -0.7262943421228756
LBFGS Accumulation History: 3 points
Removed measurement 4ef1d79 to history. Total: 3
th(0)=-18.26286524437022;dx=-3.1898590531311834E-4
New Minimum: -18.26286524437022 > -19.59972341102902
END: th(6981.964273251479)=-19.59972341102902; dx=-6.013930137398465E-5 evalInputDelta=1.3368581666588
Fitness changed from -18.26286524437022 to -19.59972341102902
Iteration 13 complete. Error: -19.59972341102902 Total: 0.3685; Orientation: 0.0287; Line Search: 0.2261
Adding measurement 374ccf4a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.691e+04, gradient 1.440e-02, dot -0.499; [2a5d3207-7741-4d6e-9241-975ca8d3ac4a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.59972341102902, -5.089785887901949, -2.583180455754794, -0.7262943421228756
LBFGS Accumulation History: 3 points
Removed measurement 374ccf4a to history. Total: 3
th(0)=-19.59972341102902;dx=-1.9870881227461836E-4
New Minimum: -19.59972341102902 > -21.1194577327939
END: th(15042.186034856239)=-21.1194577327939; dx=-2.6664643355954774E-5 evalInputDelta=1.5197343217648793
Fitness changed from -19.59972341102902 to -21.1194577327939
Iteration 14 complete. Error: -21.1194577327939 Total: 0.3688; Orientation: 0.0280; Line Search: 0.2297
Adding measurement 2b8d3a08 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.221e+04, gradient 1.940e-02, dot -0.385; [2a5d3207-7741-4d6e-9241-975ca8d3ac4a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -21.1194577327939, -5.089785887901949, -2.583180455754794, -0.7262943421228756
LBFGS Accumulation History: 3 points
Removed measurement 2b8d3a08 to history. Total: 3
th(0)=-21.1194577327939;dx=-3.6777063510334905E-4
Armijo: th(32407.40740740743)=-16.78876161159311; dx=5.804496453198547E-5 evalInputDelta=-4.330696121200791
Armijo: th(16203.703703703715)=-20.847936374291717; dx=1.2760383831095581E-4 evalInputDelta=-0.2715213585021843
New Minimum: -21.1194577327939 > -22.623121102080248
END: th(5401.234567901239)=-22.623121102080248; dx=-8.681134283902169E-5 evalInputDelta=1.5036633692863468
Fitness changed from -21.1194577327939 to -22.623121102080248
Iteration 15 complete. Error: -22.623121102080248 Total: 0.5968; Orientation: 0.0297; Line Search: 0.4576
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -22.623121102080248 (> -Infinity) after 8.388s (< 5400.000s)
Returns
-22.623121102080248