BasicOptimizer.scala:89 executed in 50.11 seconds (0.573 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 3403209415054
Reset training subject: 3403476730767
Adding measurement 450f4700 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-69.65186168321347;dx=-8.906953863229896E-5
New Minimum: -69.65186168321347 > -69.65187219867998
WOLFE (weak): th(2.154434690031884)=-69.65187219867998; dx=-8.904053785053007E-5 evalInputDelta=1.051546651353874E-5
New Minimum: -69.65187219867998 > -69.65188271414492
WOLFE (weak): th(4.308869380063768)=-69.65188271414492; dx=-8.904096501745902E-5 evalInputDelta=2.1030931449672607E-5
New Minimum: -69.65188271414492 > -69.65191776568324
WOLFE (weak): th(12.926608140191302)=-69.65191776568324; dx=-8.904575098929782E-5 evalInputDelta=5.608246976862574E-5
New Minimum: -69.65191776568324 > -69.65207900253229
WOLFE (weak): th(51.70643256076521)=-69.65207900253229; dx=-8.904599444007672E-5 evalInputDelta=2.173193188212963E-4
New Minimum: -69.65207900253229 > -69.65296229343014
WOLFE (weak): th(258.53216280382605)=-69.65296229343014; dx=-8.903039477380713E-5 evalInputDelta=0.0011006102166675191
New Minimum: -69.65296229343014 > -69.6585176559999
WOLFE (weak): th(1551.1929768229563)=-69.6585176559999; dx=-8.880504039302016E-5 evalInputDelta=0.00665597278643304
New Minimum: -69.6585176559999 > -69.70001639279756
WOLFE (weak): th(10858.350837760694)=-69.70001639279756; dx=-8.714632409467043E-5 evalInputDelta=0.04815470958409662
New Minimum: -69.70001639279756 > -70.08575592510756
END: th(86866.80670208555)=-70.08575592510756; dx=-7.486679040179084E-5 evalInputDelta=0.43389424189409453
Fitness changed from -69.65186168321347 to -70.08575592510756
Iteration 1 complete. Error: -70.08575592510756 Total: 3.5657; Orientation: 0.0163; Line Search: 2.7583
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 63f73b5c to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-70.08575592510756;dx=-8.133525204718997E-5
New Minimum: -70.08575592510756 > -71.01051789941931
END: th(187148.86177126726)=-71.01051789941931; dx=-5.510130275990203E-5 evalInputDelta=0.9247619743117497
Fitness changed from -70.08575592510756 to -71.01051789941931
Iteration 2 complete. Error: -71.01051789941931 Total: 0.9006; Orientation: 0.0166; Line Search: 0.6072
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 4b80f6ed to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-71.01051789941931;dx=-6.342868558284413E-5
New Minimum: -71.01051789941931 > -72.05698428587613
END: th(403200.00000000006)=-72.05698428587613; dx=-3.1404215983218065E-5 evalInputDelta=1.0464663864568138
Fitness changed from -71.01051789941931 to -72.05698428587613
Iteration 3 complete. Error: -72.05698428587613 Total: 0.9310; Orientation: 0.0162; Line Search: 0.6217
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 387063bc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.762e+04, gradient 6.158e-03, dot -0.602; [5e3d83b0-ec32-4983-b7ef-88d3aad26226 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -72.05698428587613, -71.01051789941931, -70.08575592510756, -69.65186168321347
LBFGS Accumulation History: 3 points
Removed measurement 387063bc to history. Total: 3
th(0)=-72.05698428587613;dx=-3.787271656910872E-5
New Minimum: -72.05698428587613 > -72.57590315490397
END: th(868668.0670208557)=-72.57590315490397; dx=-1.5382696488652274E-5 evalInputDelta=0.5189188690278428
Fitness changed from -72.05698428587613 to -72.57590315490397
Iteration 4 complete. Error: -72.57590315490397 Total: 1.3047; Orientation: 0.4243; Line Search: 0.6143
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 5fd05b10 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.353e+04, gradient 4.424e-03, dot -0.485; [5e3d83b0-ec32-4983-b7ef-88d3aad26226 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -72.57590315490397, -71.01051789941931, -70.08575592510756, -69.65186168321347
LBFGS Accumulation History: 3 points
Removed measurement 5fd05b10 to history. Total: 3
th(0)=-72.57590315490397;dx=-1.9550436590035433E-5
New Minimum: -72.57590315490397 > -72.840136922407
END: th(1871488.617712673)=-72.840136922407; dx=-7.01836529381286E-6 evalInputDelta=0.26423376750302907
Fitness changed from -72.57590315490397 to -72.840136922407
Iteration 5 complete. Error: -72.840136922407 Total: 1.7904; Orientation: 0.8557; Line Search: 0.6482
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 5be11517 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.172e+04, gradient 3.079e-03, dot -0.446; [5e3d83b0-ec32-4983-b7ef-88d3aad26226 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -72.840136922407, -71.01051789941931, -70.08575592510756, -69.65186168321347
LBFGS Accumulation History: 3 points
Removed measurement 5be11517 to history. Total: 3
th(0)=-72.840136922407;dx=-9.469996948299973E-6
Armijo: th(4032000.0000000014)=-72.73218478173882; dx=-3.075301136609167E-6 evalInputDelta=-0.10795214066817493
New Minimum: -72.840136922407 > -72.89558363358681
END: th(2016000.0000000007)=-72.89558363358681; dx=-4.9527651554685695E-6 evalInputDelta=0.055446711179811814
Fitness changed from -72.840136922407 to -72.89558363358681
Iteration 6 complete. Error: -72.89558363358681 Total: 1.6329; Orientation: 0.4269; Line Search: 0.9325
Adding measurement 1728a570 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.300e+04, gradient 2.586e-03, dot -0.480; [5e3d83b0-ec32-4983-b7ef-88d3aad26226 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -72.89558363358681, -71.01051789941931, -70.08575592510756, -69.65186168321347
LBFGS Accumulation History: 3 points
Removed measurement 1728a570 to history. Total: 3
th(0)=-72.89558363358681;dx=-6.679049346533456E-6
Armijo: th(4343340.335104279)=-72.80236986935076; dx=-2.617831819468991E-6 evalInputDelta=-0.09321376423605443
Armijo: th(2171670.1675521396)=-72.88585025894086; dx=-3.9034653390021215E-6 evalInputDelta=-0.009733374645946924
New Minimum: -72.89558363358681 > -72.89597213774057
END: th(723890.0558507132)=-72.89597213774057; dx=-5.246244727605595E-6 evalInputDelta=3.8850415376145975E-4
Fitness changed from -72.89558363358681 to -72.89597213774057
Iteration 7 complete. Error: -72.89597213774057 Total: 1.9338; Orientation: 0.4154; Line Search: 1.2598
Adding measurement 21092b13 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.382e+04, gradient 2.480e-03, dot -0.499; [5e3d83b0-ec32-4983-b7ef-88d3aad26226 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -72.89597213774057, -71.01051789941931, -70.08575592510756, -69.65186168321347
LBFGS Accumulation History: 3 points
Removed measurement 21092b13 to history. Total: 3
th(0)=-72.89597213774057;dx=-6.144682385133613E-6
New Minimum: -72.89597213774057 > -72.90488706934364
END: th(1559573.8480938943)=-72.90488706934364; dx=-4.076158618422716E-6 evalInputDelta=0.008914931603072773
Fitness changed from -72.89597213774057 to -72.90488706934364
Iteration 8 complete. Error: -72.90488706934364 Total: 1.3029; Orientation: 0.4234; Line Search: 0.6422
Adding measurement 3fdf2bc6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.473e+04, gradient 2.286e-03, dot -0.515; [5e3d83b0-ec32-4983-b7ef-88d3aad26226 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -72.90488706934364, -71.01051789941931, -70.08575592510756, -69.65186168321347
LBFGS Accumulation History: 3 points
Removed measurement 3fdf2bc6 to history. Total: 3
th(0)=-72.90488706934364;dx=-5.2203563187045656E-6
New Minimum: -72.90488706934364 > -72.92579035181929
END: th(3360000.0000000014)=-72.92579035181929; dx=-2.6493002032025563E-6 evalInputDelta=0.020903282475643437
Fitness changed from -72.90488706934364 to -72.92579035181929
Iteration 9 complete. Error: -72.92579035181929 Tot
...skipping 10853 bytes...
1, -70.08575592510756, -69.65186168321347
LBFGS Accumulation History: 3 points
Removed measurement 5106802d to history. Total: 3
th(0)=-74.30438809278864;dx=-2.0510255798641917E-7
Armijo: th(3.7702607075558007E8)=-74.0760832744365; dx=3.1868577158403176E-9 evalInputDelta=-0.228304818352143
Armijo: th(1.8851303537779003E8)=-74.284020641429; dx=-2.3678670563123872E-9 evalInputDelta=-0.020367451359646793
New Minimum: -74.30438809278864 > -74.37871905092376
END: th(6.283767845926335E7)=-74.37871905092376; dx=-2.682661967272675E-8 evalInputDelta=0.07433095813512125
Fitness changed from -74.30438809278864 to -74.37871905092376
Iteration 22 complete. Error: -74.37871905092376 Total: 2.1667; Orientation: 0.4365; Line Search: 1.2724
Adding measurement 555f421a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.160e+04, gradient 4.438e-04, dot -0.158; [5e3d83b0-ec32-4983-b7ef-88d3aad26226 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -74.37871905092376, -71.01051789941931, -70.08575592510756, -69.65186168321347
LBFGS Accumulation History: 3 points
Removed measurement 555f421a to history. Total: 3
th(0)=-74.37871905092376;dx=-1.9682170682208585E-7
Armijo: th(1.3537967431370622E8)=-74.349811911329; dx=-7.364346598060963E-9 evalInputDelta=-0.02890713959476443
New Minimum: -74.37871905092376 > -74.38407245086141
END: th(6.768983715685311E7)=-74.38407245086141; dx=-2.0018862050866904E-8 evalInputDelta=0.005353399937646941
Fitness changed from -74.37871905092376 to -74.38407245086141
Iteration 23 complete. Error: -74.38407245086141 Total: 1.6449; Orientation: 0.4144; Line Search: 0.9626
Adding measurement 55023db2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.740e+04, gradient 4.381e-04, dot -0.162; [5e3d83b0-ec32-4983-b7ef-88d3aad26226 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -74.38407245086141, -71.01051789941931, -70.08575592510756, -69.65186168321347
LBFGS Accumulation History: 3 points
Removed measurement 55023db2 to history. Total: 3
th(0)=-74.38407245086141;dx=-1.9168816367142838E-7
Armijo: th(1.4583333333333352E8)=-74.36858887809254; dx=-4.958009465841826E-9 evalInputDelta=-0.015483572768872023
New Minimum: -74.38407245086141 > -74.39205096039328
END: th(7.291666666666676E7)=-74.39205096039328; dx=-1.512514008141967E-8 evalInputDelta=0.007978509531866962
Fitness changed from -74.38407245086141 to -74.39205096039328
Iteration 24 complete. Error: -74.39205096039328 Total: 1.6129; Orientation: 0.4161; Line Search: 0.9430
Adding measurement 160df2e6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.294e+04, gradient 4.347e-04, dot -0.161; [5e3d83b0-ec32-4983-b7ef-88d3aad26226 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -74.39205096039328, -71.01051789941931, -70.08575592510756, -69.65186168321347
LBFGS Accumulation History: 3 points
Removed measurement 160df2e6 to history. Total: 3
th(0)=-74.39205096039328;dx=-1.8816859908100544E-7
Armijo: th(1.570941961481584E8)=-74.35602436504388; dx=-3.2475422602911195E-9 evalInputDelta=-0.036026595349397894
New Minimum: -74.39205096039328 > -74.40238138304949
END: th(7.85470980740792E7)=-74.40238138304949; dx=-1.2245358630787777E-8 evalInputDelta=0.010330422656210203
Fitness changed from -74.39205096039328 to -74.40238138304949
Iteration 25 complete. Error: -74.40238138304949 Total: 1.6445; Orientation: 0.4155; Line Search: 0.9684
<a id="p-19"></a>Iteration 25
<a id="p-18"></a>
Adding measurement 69df7ef7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.559e+04, gradient 4.316e-04, dot -0.158; [5e3d83b0-ec32-4983-b7ef-88d3aad26226 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -74.40238138304949, -71.01051789941931, -70.08575592510756, -69.65186168321347
LBFGS Accumulation History: 3 points
Removed measurement 69df7ef7 to history. Total: 3
th(0)=-74.40238138304949;dx=-1.8614462368883497E-7
Armijo: th(1.692245928921328E8)=-74.32553502161959; dx=-2.2841065260516216E-9 evalInputDelta=-0.07684636142990087
Armijo: th(8.46122964460664E7)=-74.39388875165587; dx=-1.001278977334794E-8 evalInputDelta=-0.008492631393622219
New Minimum: -74.40238138304949 > -74.40806119173178
END: th(2.8204098815355465E7)=-74.40806119173178; dx=-2.1799842129524873E-8 evalInputDelta=0.005679808682288012
Fitness changed from -74.40238138304949 to -74.40806119173178
Iteration 26 complete. Error: -74.40806119173178 Total: 1.9719; Orientation: 0.4159; Line Search: 1.2760
Adding measurement d5cccad to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.918e+04, gradient 4.268e-04, dot -0.162; [5e3d83b0-ec32-4983-b7ef-88d3aad26226 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -74.40806119173178, -71.01051789941931, -70.08575592510756, -69.65186168321347
LBFGS Accumulation History: 3 points
Removed measurement d5cccad to history. Total: 3
th(0)=-74.40806119173178;dx=-1.8194488036605987E-7
Armijo: th(6.076388888888897E7)=-74.39675036480928; dx=-1.307940781057729E-8 evalInputDelta=-0.01131082692249663
Armijo: th(3.0381944444444485E7)=-74.40574469589025; dx=-1.9181322463733037E-8 evalInputDelta=-0.002316495841526489
New Minimum: -74.40806119173178 > -74.4087206908152
END: th(1.0127314814814828E7)=-74.4087206908152; dx=-2.4940437495858614E-8 evalInputDelta=6.594990834258851E-4
Fitness changed from -74.40806119173178 to -74.4087206908152
Iteration 27 complete. Error: -74.4087206908152 Total: 1.9444; Orientation: 0.4140; Line Search: 1.2686
Adding measurement 5e3e6d7f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.069e+04, gradient 4.259e-04, dot -0.163; [5e3d83b0-ec32-4983-b7ef-88d3aad26226 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -74.4087206908152, -71.01051789941931, -70.08575592510756, -69.65186168321347
LBFGS Accumulation History: 3 points
Removed measurement 5e3e6d7f to history. Total: 3
th(0)=-74.4087206908152;dx=-1.8121836638827456E-7
Armijo: th(2.181863835391089E7)=-74.40797588296566; dx=-2.0486453437022954E-8 evalInputDelta=-7.448078495428945E-4
Armijo: th(1.0909319176955445E7)=-74.40762152242874; dx=-2.343191687660805E-8 evalInputDelta=-0.001099168386460292
Armijo: th(3636439.725651815)=-74.40823837116424; dx=-2.601981962465319E-8 evalInputDelta=-4.823196509562422E-4
Armijo: th(909109.9314129538)=-74.40862225848728; dx=-2.7176484707433834E-8 evalInputDelta=-9.8432327916953E-5
Armijo: th(181821.98628259075)=-74.40870100436004; dx=-2.746170758051202E-8 evalInputDelta=-1.9686455161149752E-5
Armijo: th(30303.66438043179)=-74.4087206908152; dx=-2.7478961516405913E-8 evalInputDelta=0.0
Armijo: th(4329.094911490256)=-74.4087206908152; dx=-2.7518107854980234E-8 evalInputDelta=0.0
Armijo: th(541.136863936282)=-74.4087206908152; dx=-2.7518697550393624E-8 evalInputDelta=0.0
Armijo: th(60.12631821514245)=-74.4087206908152; dx=-2.753724743055682E-8 evalInputDelta=0.0
Armijo: th(6.012631821514245)=-74.4087206908152; dx=-2.751714436771383E-8 evalInputDelta=0.0
Armijo: th(0.5466028928649314)=-74.4087206908152; dx=-2.7518771820032802E-8 evalInputDelta=0.0
Armijo: th(0.04555024107207761)=-74.4087206908152; dx=-2.7473115164037116E-8 evalInputDelta=0.0
END: th(0.003503864697852124)=-74.4087206908152; dx=-2.7520326404689124E-8 evalInputDelta=0.0
Fitness changed from -74.4087206908152 to -74.4087206908152
Static Iteration Total: 5.1245; Orientation: 0.4134; Line Search: 4.4468
Iteration 28 failed. Error: -74.4087206908152
Previous Error: 0.0 -> -74.4087206908152
Retrying iteration 28
Reset training subject: 3452146970246
Adding measurement 2f2bbc10 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-74.4087206908152;dx=-1.8124736838599328E-7
END: th(0.007548847654230701)=-74.4087206908152; dx=-2.752895587937727E-8 evalInputDelta=0.0
Fitness changed from -74.4087206908152 to -74.4087206908152
Static Iteration Total: 1.1706; Orientation: 0.0168; Line Search: 0.6393
Iteration 29 failed. Error: -74.4087206908152
Previous Error: 0.0 -> -74.4087206908152
Optimization terminated 29
Final threshold in iteration 29: -74.4087206908152 (> -Infinity) after 50.108s (< 3600.000s)
Returns
-74.4087206908152