BasicOptimizer.scala:89 executed in 8.80 seconds (0.368 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 152293575599900
Reset training subject: 152293705863200
Adding measurement 2c84299a to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-52.532799928800436;dx=-0.2844202406413994
New Minimum: -52.532799928800436 > -53.598883682696695
WOLFE (weak): th(2.154434690031884)=-53.598883682696695; dx=-0.289408603888427 evalInputDelta=1.0660837538962582
New Minimum: -53.598883682696695 > -54.672145165808615
WOLFE (weak): th(4.308869380063768)=-54.672145165808615; dx=-0.2931661706190982 evalInputDelta=2.139345237008179
New Minimum: -54.672145165808615 > -58.92047656455055
WOLFE (weak): th(12.926608140191302)=-58.92047656455055; dx=-0.28790174729849516 evalInputDelta=6.387676635750111
New Minimum: -58.92047656455055 > -77.3863834518063
WOLFE (weak): th(51.70643256076521)=-77.3863834518063; dx=-0.2653248249451885 evalInputDelta=24.85358352300586
New Minimum: -77.3863834518063 > -128.20124255930594
END: th(258.53216280382605)=-128.20124255930594; dx=-0.060776353754553085 evalInputDelta=75.6684426305055
Fitness changed from -52.532799928800436 to -128.20124255930594
Iteration 1 complete. Error: -128.20124255930594 Total: 1.0388; Orientation: 0.0016; Line Search: 0.6913
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 622fa131 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-128.20124255930594;dx=-0.1893454637697796
New Minimum: -128.20124255930594 > -213.6189839702923
END: th(556.9906600335335)=-213.6189839702923; dx=-0.02857463897078355 evalInputDelta=85.41774141098637
Fitness changed from -128.20124255930594 to -213.6189839702923
Iteration 2 complete. Error: -213.6189839702923 Total: 0.3434; Orientation: 0.0015; Line Search: 0.2236
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 39a40ba7 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-213.6189839702923;dx=-0.20375590502269864
New Minimum: -213.6189839702923 > -298.4024521305078
END: th(1200.0)=-298.4024521305078; dx=-0.006325767368297553 evalInputDelta=84.7834681602155
Fitness changed from -213.6189839702923 to -298.4024521305078
Iteration 3 complete. Error: -298.4024521305078 Total: 0.3522; Orientation: 0.0013; Line Search: 0.2314
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 35bbf1c7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.047e+03, gradient 5.307e-01, dot -0.403; [c7b76b8d-2fb5-4279-9ec7-f0a06e703460 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -298.4024521305078, -213.6189839702923, -128.20124255930594, -52.532799928800436
LBFGS Accumulation History: 3 points
Removed measurement 35bbf1c7 to history. Total: 3
th(0)=-298.4024521305078;dx=-0.26970564109677914
New Minimum: -298.4024521305078 > -367.8094572941267
END: th(2585.3216280382608)=-367.8094572941267; dx=-0.007243744474418928 evalInputDelta=69.40700516361886
Fitness changed from -298.4024521305078 to -367.8094572941267
Iteration 4 complete. Error: -367.8094572941267 Total: 0.3909; Orientation: 0.0310; Line Search: 0.2380
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 2499cfad to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.126e+03, gradient 4.226e-01, dot -0.245; [c7b76b8d-2fb5-4279-9ec7-f0a06e703460 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -367.8094572941267, -213.6189839702923, -128.20124255930594, -52.532799928800436
LBFGS Accumulation History: 3 points
Removed measurement 2499cfad to history. Total: 3
th(0)=-367.8094572941267;dx=-0.16922369351700828
New Minimum: -367.8094572941267 > -623.7128245434753
END: th(5569.9066003353355)=-623.7128245434753; dx=-0.010530725492218165 evalInputDelta=255.90336724934866
Fitness changed from -367.8094572941267 to -623.7128245434753
Iteration 5 complete. Error: -623.7128245434753 Total: 0.4018; Orientation: 0.0337; Line Search: 0.2423
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 6861ec06 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.359e+03, gradient 4.279e-01, dot -0.240; [c7b76b8d-2fb5-4279-9ec7-f0a06e703460 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -623.7128245434753, -213.6189839702923, -128.20124255930594, -52.532799928800436
LBFGS Accumulation History: 3 points
Removed measurement 6861ec06 to history. Total: 3
th(0)=-623.7128245434753;dx=-0.17670024811221632
New Minimum: -623.7128245434753 > -1105.2922690401847
WOLF (strong): th(12000.000000000002)=-1105.2922690401847; dx=0.007178655803702214 evalInputDelta=481.57944449670936
New Minimum: -1105.2922690401847 > -1196.8554110668506
END: th(6000.000000000001)=-1196.8554110668506; dx=-0.012726276178943043 evalInputDelta=573.1425865233753
Fitness changed from -623.7128245434753 to -1196.8554110668506
Iteration 6 complete. Error: -1196.8554110668506 Total: 0.5068; Orientation: 0.0275; Line Search: 0.3625
Adding measurement 49b3d3a9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.571e+04, gradient 3.858e-01, dot -0.313; [c7b76b8d-2fb5-4279-9ec7-f0a06e703460 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1196.8554110668506, -213.6189839702923, -128.20124255930594, -52.532799928800436
LBFGS Accumulation History: 3 points
Removed measurement 49b3d3a9 to history. Total: 3
th(0)=-1196.8554110668506;dx=-0.14149643804080403
New Minimum: -1196.8554110668506 > -1473.2178216407783
WOLF (strong): th(12926.608140191305)=-1473.2178216407783; dx=0.008887819756764243 evalInputDelta=276.3624105739277
New Minimum: -1473.2178216407783 > -1656.9075864392678
END: th(6463.304070095653)=-1656.9075864392678; dx=-0.01470767152837605 evalInputDelta=460.0521753724172
Fitness changed from -1196.8554110668506 to -1656.9075864392678
Iteration 7 complete. Error: -1656.9075864392678 Total: 0.8664; Orientation: 0.0328; Line Search: 0.7234
Adding measurement 253d7c21 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.128e+05, gradient 3.899e-01, dot -0.396; [c7b76b8d-2fb5-4279-9ec7-f0a06e703460 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1656.9075864392678, -213.6189839702923, -128.20124255930594, -52.532799928800436
LBFGS Accumulation History: 3 points
Removed measurement 253d7c21 to history. Total: 3
th(0)=-1656.9075864392678;dx=-0.1413534767845648
New Minimum: -1656.9075864392678 > -1942.7057806060084
WOLF (strong): th(13924.766500838341)=-1942.7057806060084; dx=0.022108708293970866 evalInputDelta=285.7981941667406
New Minimum: -1942.7057806060084 > -2226.8720214686787
WOLF (strong): th(6962.3832504191705)=-2226.8720214686787; dx=0.0022243162300904958 evalInputDelta=569.9644350294109
END: th(2320.79441680639)=-2053.4259178261095; dx=-0.06743709502816564 evalInputDelta=396.51833138684174
Fitness changed from -1656.9075864392678 to -2226.8720214686787
Iteration 8 complete. Error: -2226.8720214686787 Total: 0.6121; Orientation: 0.0296; Line Search: 0.4598
Adding measurement 7b37e915 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.970e+05, gradient 4.154e-01, dot -0.351; [c7b76b8d-2fb5-4279-9ec7-f0a06e703460 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2226.8720214686787, -213.6189839702923, -128.20124255930594, -52.532799928800436
LBFGS Accumulation History: 3 points
Removed measurement 7b37e915 to history. Total: 3
th(0)=-2226.8720214686787;dx=-0.15858305079878993
New Minimum: -2226.8720214686787 > -2611.872795524315
END: th(5000.000000000002)=-2611.872795524315; dx=-0.015398620467756974 evalInputDelta=385.00077405563616
Fitness changed from -2226.8720214686787 to -2611.872795524315
Iteration 9 complete. Error: -2611.872795524315 Total: 0.3731; Orientation: 0.0316; Line Search: 0.2301
Adding measurement 7bea637e to history. Total: 3
Accepted: LBFGS Orientation magnitude: 2.946e+04, gradient 3.898e-01, dot 0.481; [c7b76b8d-2fb5-4279-9ec7-f0a06e703460 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-2611.872795524315;dx=-5676.460958237547
New Minimum: -2611.872795524315 > -3121.8625850603994
WOLF (strong): th(10772.173450159424)=-3121.8625850603994; dx=0.04941680212418541 evalInputDelta=509.9897895360846
WOLF (strong): th(5386.086725079712)=-3121.8054071322254; dx=0.09681694688618553 evalInputDelta=509.93261160791053
WOLF (strong): th(1795.3622416932374)=-3120.9025938019918; dx=0.29073593497505124 evalInputDelta=509.02979827767695
WOLF (strong): th(448.84056042330934)=-3120.89458328858; dx=1.1360759559959843 evalInputDelta=509.02178776426535
New Minimum: -3121.8625850603994 > -3125.7074399246003
WOLF (strong): th(89.76811208466187)=-3125.7074399246003; dx=5.739207603600764 evalInputDelta=513.8346444002855
New Minimum: -3125.7074399246003 > -3131.713907750834
WOLF (strong): th(14.961352014110311)=-3131.713907750834; dx=33.051431948651064 evalInputDelta=519.8411122265193
New Minimum: -3131.713907750834 > -3286.956647112949
WOLF (strong): th(2.1373360020157586)=-3286.956647112949; dx=106.29023201832935 evalInputDelta=675.0838515886344
New Minimum: -3286.956647112949 > -3287.721095226905
END: th(0.2671670002519698)=-3287.721095226905; dx=-1088.1532134833858 evalInputDelta=675.8482997025903
Fitness changed from -2611.872795524315 to -3287.721095226905
Iteration 10 complete. Error: -3287.721095226905 Total: 1.1685; Orientation: 0.0258; Line Search: 1.0321
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 45daa94 to history. Total: 4
Accepted: LBFGS Orientation magnitude: 2.349e+04, gradient 3.763e-01, dot 0.421; [c7b76b8d-2fb5-4279-9ec7-f0a06e703460 = 1.000/1.000e+00]
th(0)=-3287.721095226905;dx=-3837.9008023018746
New Minimum: -3287.721095226905 > -3395.926088712768
END: th(0.5755938533746008)=-3395.926088712768; dx=-134.19997314399987 evalInputDelta=108.20499348586281
Fitness changed from -3287.721095226905 to -3395.926088712768
Iteration 11 complete. Error: -3395.926088712768 Total: 0.3835; Orientation: 0.0326; Line Search: 0.2333
Adding measurement 74994967 to history. Total: 5
Accepted: LBFGS Orientation magnitude: 1.863e+04, gradient 3.600e-01, dot 0.348; [c7b76b8d-2fb5-4279-9ec7-f0a06e703460 = 1.000/1.000e+00]
th(0)=-3395.926088712768;dx=-2330.85116100558
Armijo: th(1.2400793650793656)=-3339.0011979632473; dx=13.655365155725423 evalInputDelta=-56.92489074952073
Armijo: th(0.6200396825396828)=-3373.590668708935; dx=-13.596231795757912 evalInputDelta=-22.33542000383295
Armijo: th(0.20667989417989427)=-3390.519429232046; dx=-50.80150400009818 evalInputDelta=-5.406659480721828
Armijo: th(0.05166997354497357)=-3394.999263622895; dx=-66.17801076057476 evalInputDelta=-0.9268250898730912
Armijo: th(0.010333994708994713)=-3395.771488189392; dx=-81.46493555363276 evalInputDelta=-0.15460052337584784
Armijo: th(0.0017223324514991187)=-3395.902089283494; dx=-91.56331550998335 evalInputDelta=-0.023999429273771966
Armijo: th(2.460474930713027E-4)=-3395.92270230051; dx=-78.67054935113683 evalInputDelta=-0.003386412257896154
Armijo: th(3.075593663391284E-5)=-3395.925058065917; dx=-86.5085166338702 evalInputDelta=-0.0010306468511771527
Armijo: th(3.417326292656982E-6)=-3395.926088712768; dx=-90.42378472223032 evalInputDelta=0.0
Armijo: th(3.417326292656982E-7)=-3395.926088712768; dx=-85.13489060339973 evalInputDelta=0.0
Armijo: th(3.1066602660518015E-8)=-3395.926088712768; dx=-84.25174432914731 evalInputDelta=0.0
Armijo: th(2.5888835550431677E-9)=-3395.926088712768; dx=-83.39738681662838 evalInputDelta=0.0
Armijo: th(1.9914488884947445E-10)=-3395.926088712768; dx=-84.44253428529336 evalInputDelta=0.0
MIN ALPHA (1.4224634917819604E-11): th(0.0)=-3395.926088712768
Fitness changed from -3395.926088712768 to -3395.926088712768
Static Iteration Total: 1.9070; Orientation: 0.0425; Line Search: 1.7576
Iteration 12 failed. Error: -3395.926088712768
Previous Error: 0.0 -> -3395.926088712768
Retrying iteration 12
Reset training subject: 152301920191100
Adding measurement 7ab244b6 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-3395.926088712768;dx=-0.12332783523861413
END: th(2.1544346900318838E-10)=-3395.926088712768; dx=-0.06180258951250124 evalInputDelta=0.0
Fitness changed from -3395.926088712768 to -3395.926088712768
Static Iteration Total: 0.4523; Orientation: 0.0014; Line Search: 0.2218
Iteration 13 failed. Error: -3395.926088712768
Previous Error: 0.0 -> -3395.926088712768
Optimization terminated 13
Final threshold in iteration 13: -3395.926088712768 (> -Infinity) after 8.797s (< 5400.000s)
Returns
-3395.926088712768