BasicOptimizer.scala:89 executed in 62.72 seconds (1.592 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 139491668435900
Reset training subject: 139492575401300
Adding measurement 6bfdc031 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-10.600270983310635;dx=-0.001317605907891423
New Minimum: -10.600270983310635 > -10.603726560541437
WOLFE (weak): th(2.154434690031884)=-10.603726560541437; dx=-0.0013165258571760773 evalInputDelta=0.0034555772308024757
New Minimum: -10.603726560541437 > -10.607179277273177
WOLFE (weak): th(4.308869380063768)=-10.607179277273177; dx=-0.0013186967200734536 evalInputDelta=0.006908293962542444
New Minimum: -10.607179277273177 > -10.620993646171698
WOLFE (weak): th(12.926608140191302)=-10.620993646171698; dx=-0.0013171274684980435 evalInputDelta=0.020722662861063057
New Minimum: -10.620993646171698 > -10.683054801721871
WOLFE (weak): th(51.70643256076521)=-10.683054801721871; dx=-0.001315554815014911 evalInputDelta=0.08278381841123661
New Minimum: -10.683054801721871 > -11.012714581064296
WOLFE (weak): th(258.53216280382605)=-11.012714581064296; dx=-0.0013058376132656934 evalInputDelta=0.41244359775366135
New Minimum: -11.012714581064296 > -12.959670032593849
WOLFE (weak): th(1551.1929768229563)=-12.959670032593849; dx=-0.0011908509860126073 evalInputDelta=2.3593990492832138
New Minimum: -12.959670032593849 > -19.67372256142054
END: th(10858.350837760694)=-19.67372256142054; dx=-3.157726059926458E-4 evalInputDelta=9.073451578109905
Fitness changed from -10.600270983310635 to -19.67372256142054
Iteration 1 complete. Error: -19.67372256142054 Total: 10.6799; Orientation: 0.0159; Line Search: 8.1004
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 561c9066 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-19.67372256142054;dx=-0.0011926885340743103
New Minimum: -19.67372256142054 > -29.607526438277965
END: th(23393.607721408407)=-29.607526438277965; dx=-1.2606200684437443E-4 evalInputDelta=9.933803876857425
Fitness changed from -19.67372256142054 to -29.607526438277965
Iteration 2 complete. Error: -29.607526438277965 Total: 2.7110; Orientation: 0.0158; Line Search: 1.7462
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 6194fb9a to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-29.607526438277965;dx=-0.0014646451885071956
New Minimum: -29.607526438277965 > -34.42403519088494
WOLF (strong): th(50400.00000000001)=-34.42403519088494; dx=1.2172953789321918E-4 evalInputDelta=4.816508752606978
New Minimum: -34.42403519088494 > -38.67221666533499
END: th(25200.000000000004)=-38.67221666533499; dx=-5.177089312374114E-6 evalInputDelta=9.064690227057028
Fitness changed from -29.607526438277965 to -38.67221666533499
Iteration 3 complete. Error: -38.67221666533499 Total: 3.7843; Orientation: 0.0178; Line Search: 2.6555
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 485d410c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.776e+03, gradient 3.731e-02, dot -0.347; [62433f20-46a2-4473-9ece-dcffb082ee07 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -38.67221666533499, -29.607526438277965, -19.67372256142054, -10.600270983310635
LBFGS Accumulation History: 3 points
Removed measurement 485d410c to history. Total: 3
th(0)=-38.67221666533499;dx=-0.0013893932390422743
Armijo: th(54291.75418880348)=-36.70905400602753; dx=1.654856980851947E-4 evalInputDelta=-1.9631626593074643
New Minimum: -38.67221666533499 > -46.772175634992855
WOLF (strong): th(27145.87709440174)=-46.772175634992855; dx=1.5101574734493708E-4 evalInputDelta=8.099958969657862
New Minimum: -46.772175634992855 > -48.9070190948604
END: th(9048.625698133914)=-48.9070190948604; dx=-4.5218524280286605E-4 evalInputDelta=10.23480242952541
Fitness changed from -38.67221666533499 to -48.9070190948604
Iteration 4 complete. Error: -48.9070190948604 Total: 4.8324; Orientation: 0.3264; Line Search: 3.5243
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 1bda3651 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.470e+03, gradient 2.781e-02, dot -0.465; [62433f20-46a2-4473-9ece-dcffb082ee07 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -48.9070190948604, -29.607526438277965, -19.67372256142054, -10.600270983310635
LBFGS Accumulation History: 3 points
Removed measurement 1bda3651 to history. Total: 3
th(0)=-48.9070190948604;dx=-7.724876350098722E-4
New Minimum: -48.9070190948604 > -57.63364020547034
END: th(19494.673101173677)=-57.63364020547034; dx=-2.9948162853410333E-4 evalInputDelta=8.72662111060994
Fitness changed from -48.9070190948604 to -57.63364020547034
Iteration 5 complete. Error: -57.63364020547034 Total: 3.0702; Orientation: 0.2895; Line Search: 1.8419
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 4b1d2995 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.243e+03, gradient 3.066e-02, dot -0.417; [62433f20-46a2-4473-9ece-dcffb082ee07 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -57.63364020547034, -29.607526438277965, -19.67372256142054, -10.600270983310635
LBFGS Accumulation History: 3 points
Removed measurement 4b1d2995 to history. Total: 3
th(0)=-57.63364020547034;dx=-9.334934654319586E-4
New Minimum: -57.63364020547034 > -58.30515982829998
WOLF (strong): th(42000.000000000015)=-58.30515982829998; dx=9.940392181807139E-5 evalInputDelta=0.671519622829635
New Minimum: -58.30515982829998 > -63.02700445805126
END: th(21000.000000000007)=-63.02700445805126; dx=-3.217066622644733E-5 evalInputDelta=5.393364252580916
Fitness changed from -57.63364020547034 to -63.02700445805126
Iteration 6 complete. Error: -63.02700445805126 Total: 3.9144; Orientation: 0.2924; Line Search: 2.6673
Adding measurement 2d23de9b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.162e+03, gradient 3.570e-02, dot -0.376; [62433f20-46a2-4473-9ece-dcffb082ee07 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -63.02700445805126, -29.607526438277965, -19.67372256142054, -10.600270983310635
LBFGS Accumulation History: 3 points
Removed measurement 2d23de9b to history. Total: 3
th(0)=-63.02700445805126;dx=-0.0012704065044885178
Armijo: th(45243.12849066957)=-61.20718884848794; dx=1.7066678315661907E-4 evalInputDelta=-1.819815609563321
New Minimum: -63.02700445805126 > -69.17730232664158
WOLF (strong): th(22621.564245334786)=-69.17730232664158; dx=3.7305204321299906E-5 evalInputDelta=6.150297868590322
New Minimum: -69.17730232664158 > -70.53658631676998
END: th(7540.521415111595)=-70.53658631676998; dx=-5.025681771298274E-4 evalInputDelta=7.509581858718725
Fitness changed from -63.02700445805126 to -70.53658631676998
Iteration 7 complete. Error: -70.53658631676998 Total: 4.5882; Orientation: 0.2881; Line Search: 3.4286
Adding measurement 49639e24 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.210e+03, gradient 2.615e-02, dot -0.533; [62433f20-46a2-4473-9ece-dcffb082ee07 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -70.53658631676998, -29.607526438277965, -19.67372256142054, -10.600270983310635
LBFGS Accumulation History: 3 points
Removed measurement 49639e24 to history. Total: 3
th(0)=-70.53658631676998;dx=-6.80227389558393E-4
New Minimum: -70.53658631676998 > -77.75612798279559
END: th(16245.56091764473)=-77.75612798279559; dx=-3.7182695275907033E-4 evalInputDelta=7.219541666025606
Fitness changed from -70.53658631676998 to -77.75612798279559
Iteration 8 complete. Error: -77.75612798279559 Total: 3.0581; Orientation: 0.2920; Line Search: 1.9535
Adding measurement 470a265a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.320e+03, gradient 2.623e-02, dot -0.507; [62433f20-46a2-4473-9ece-dcffb082ee07 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -77.75612798279559, -29.607526438277965, -19.67372256142054, -10.600270983310635
LBFGS Accumulation History: 3 points
Removed measurement 470a265a to history. Total: 3
th(0)=-77.75612798279559;dx=-6.851474632087497E-4
New Minimum: -77.75612798279559 > -84.36268421598199
END: th(35000.000000000015)=-84.36268421598199; dx=-1.686891841866037E-4 evalInputDelta=6.606556233186396
Fitness changed from -77.75612798279559 to -84.36268421598199
Iteration 9 complete. Error: -84.36268421598199 Total: 2.9114; Orientation: 0.2895; Line Search: 1.8053
Adding measurement 5155bef1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.606e+03, gradient 3.395e-02, dot -0.379; [62433f20-46a2-4473-9ece-dcffb082ee07 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -84.36268421598199, -29.607526438277965, -19.67372256142054, -10.600270983310635
LBFGS Accumulation History: 3 points
Removed measurement 5155bef1 to history. Total: 3
th(0)=-84.36268421598199;dx=-0.0011451663909480471
Armijo: th(75405.21415111597)=-71.86103306664687; dx=1.3408003542432874E-4 evalInputDelta=-12.501651149335117
New Minimum: -84.36268421598199 > -88.34490366020255
WOLF (strong): th(37702.607075557986)=-88.34490366020255; dx=1.1302263880811494E-4 evalInputDelta=3.9822194442205614
New Minimum: -88.34490366020255 > -93.766121660585
END: th(12567.535691852661)=-93.766121660585; dx=-3.3068743169146277E-4 evalInputDelta=9.403437444603014
Fitness changed from -84.36268421598199 to -93.766121660585
Iteration 10 complete. Error: -93.766121660585 Total: 4.5973; Orientation: 0.2952; Line Search: 3.4710
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 1266fda5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.478e+04, gradient 2.736e-02, dot -0.488; [62433f20-46a2-4473-9ece-dcffb082ee07 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -93.766121660585, -29.607526438277965, -19.67372256142054, -10.600270983310635
LBFGS Accumulation History: 3 points
Removed measurement 1266fda5 to history. Total: 3
th(0)=-93.766121660585;dx=-7.421650767875974E-4
New Minimum: -93.766121660585 > -101.38997345428878
END: th(27075.934862741226)=-101.38997345428878; dx=-1.734609319053825E-4 evalInputDelta=7.623851793703778
Fitness changed from -93.766121660585 to -101.38997345428878
Iteration 11 complete. Error: -101.38997345428878 Total: 3.2118; Orientation: 0.2865; Line Search: 1.9480
Adding measurement 41608089 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.727e+04, gradient 3.303e-02, dot -0.382; [62433f20-46a2-4473-9ece-dcffb082ee07 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -101.38997345428878, -29.607526438277965, -19.67372256142054, -10.600270983310635
LBFGS Accumulation History: 3 points
Removed measurement 41608089 to history. Total: 3
th(0)=-101.38997345428878;dx=-0.0010869028495889244
Armijo: th(58333.33333333337)=-91.33945815773033; dx=1.831705654454451E-4 evalInputDelta=-10.050515296558444
New Minimum: -101.38997345428878 > -104.72081225405238
WOLF (strong): th(29166.666666666686)=-104.72081225405238; dx=1.0925340629019138E-4 evalInputDelta=3.3308387997635975
New Minimum: -104.72081225405238 > -108.78124636346483
END: th(9722.222222222228)=-108.78124636346483; dx=-4.064664188213803E-4 evalInputDelta=7.3912729091760525
Fitness changed from -101.38997345428878 to -108.78124636346483
Iteration 12 complete. Error: -108.78124636346483 Total: 4.5962; Orientation: 0.2869; Line Search: 3.4774
Adding measurement 21ea968b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.433e+04, gradient 2.568e-02, dot -0.536; [62433f20-46a2-4473-9ece-dcffb082ee07 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -108.78124636346483, -29.607526438277965, -19.67372256142054, -10.600270983310635
LBFGS Accumulation History: 3 points
Removed measurement 21ea968b to history. Total: 3
th(0)=-108.78124636346483;dx=-6.565582955795415E-4
New Minimum: -108.78124636346483 > -116.12330658259363
END: th(20945.892819754437)=-116.12330658259363; dx=-3.1757056752618727E-4 evalInputDelta=7.342060219128797
Fitness changed from -108.78124636346483 to -116.12330658259363
Iteration 13 complete. Error: -116.12330658259363 Total: 2.8660; Orientation: 0.2853; Line Search: 1.7681
Adding measurement 68894077 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.626e+04, gradient 2.871e-02, dot -0.471; [62433f20-46a2-4473-9ece-dcffb082ee07 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -116.12330658259363, -29.607526438277965, -19.67372256142054, -10.600270983310635
LBFGS Accumulation History: 3 points
Removed measurement 68894077 to history. Total: 3
th(0)=-116.12330658259363;dx=-8.20190962646537E-4
New Minimum: -116.12330658259363 > -120.80920739758675
END: th(45126.558104568714)=-120.80920739758675; dx=-1.0557182119930487E-4 evalInputDelta=4.6859008149931185
Fitness changed from -116.12330658259363 to -120.80920739758675
Iteration 14 complete. Error: -120.80920739758675 Total: 3.0736; Orientation: 0.2863; Line Search: 1.9392
Adding measurement 2ffd964c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.429e+05, gradient 3.828e-02, dot -0.356; [62433f20-46a2-4473-9ece-dcffb082ee07 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -120.80920739758675, -29.607526438277965, -19.67372256142054, -10.600270983310635
LBFGS Accumulation History: 3 points
Removed measurement 2ffd964c to history. Total: 3
th(0)=-120.80920739758675;dx=-0.0014631221358635408
Armijo: th(97222.22222222229)=-95.25572938877258; dx=2.0702996893875243E-4 evalInputDelta=-25.553478008814167
New Minimum: -120.80920739758675 > -124.8287863663958
WOLF (strong): th(48611.111111111146)=-124.8287863663958; dx=2.6845152942802107E-4 evalInputDelta=4.019578968809057
New Minimum: -124.8287863663958 > -137.01344781529576
END: th(16203.703703703715)=-137.01344781529576; dx=-4.1460629582282965E-4 evalInputDelta=16.204240417709016
Fitness changed from -120.80920739758675 to -137.01344781529576
Iteration 15 complete. Error: -137.01344781529576 Total: 4.7073; Orientation: 0.2978; Line Search: 3.5759
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -137.01344781529576 (> -Infinity) after 62.716s (< 5400.000s)
Returns
-137.01344781529576