BasicOptimizer.scala:89 executed in 45.44 seconds (1.060 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 216444337864100
Reset training subject: 216445168221900
Adding measurement e9fe397 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-2.1263735214982065;dx=-4.75032756229264E-5
New Minimum: -2.1263735214982065 > -2.1264345735871144
WOLFE (weak): th(2.154434690031884)=-2.1264345735871144; dx=-4.750542587327094E-5 evalInputDelta=6.105208890794245E-5
New Minimum: -2.1264345735871144 > -2.126496130266602
WOLFE (weak): th(4.308869380063768)=-2.126496130266602; dx=-4.7389065395309064E-5 evalInputDelta=1.226087683954269E-4
New Minimum: -2.126496130266602 > -2.1267425050961286
WOLFE (weak): th(12.926608140191302)=-2.1267425050961286; dx=-4.736049773375828E-5 evalInputDelta=3.6898359792214563E-4
New Minimum: -2.1267425050961286 > -2.127851033861927
WOLFE (weak): th(51.70643256076521)=-2.127851033861927; dx=-4.7461770853516014E-5 evalInputDelta=0.0014775123637207344
New Minimum: -2.127851033861927 > -2.133792577230909
WOLFE (weak): th(258.53216280382605)=-2.133792577230909; dx=-4.745837983093951E-5 evalInputDelta=0.0074190557327025175
New Minimum: -2.133792577230909 > -2.1715467076123667
WOLFE (weak): th(1551.1929768229563)=-2.1715467076123667; dx=-4.81430769524707E-5 evalInputDelta=0.04517318611416021
New Minimum: -2.1715467076123667 > -2.465006549181704
WOLFE (weak): th(10858.350837760694)=-2.465006549181704; dx=-4.917307190701718E-5 evalInputDelta=0.33863302768349746
New Minimum: -2.465006549181704 > -4.246419402952333
END: th(86866.80670208555)=-4.246419402952333; dx=-2.611933887109177E-5 evalInputDelta=2.1200458814541268
Fitness changed from -2.1263735214982065 to -4.246419402952333
Iteration 1 complete. Error: -4.246419402952333 Total: 7.4620; Orientation: 0.0161; Line Search: 5.5023
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 21a9e411 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-4.246419402952333;dx=-7.00345723870089E-5
New Minimum: -4.246419402952333 > -6.928123099974883
END: th(187148.86177126726)=-6.928123099974883; dx=-1.1726293817258817E-5 evalInputDelta=2.6817036970225496
Fitness changed from -4.246419402952333 to -6.928123099974883
Iteration 2 complete. Error: -6.928123099974883 Total: 1.7797; Orientation: 0.0159; Line Search: 1.1629
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 51fa6154 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-6.928123099974883;dx=-5.576877072704049E-5
New Minimum: -6.928123099974883 > -7.227080931012566
END: th(403200.00000000006)=-7.227080931012566; dx=-5.118518322055534E-6 evalInputDelta=0.2989578310376828
Fitness changed from -6.928123099974883 to -7.227080931012566
Iteration 3 complete. Error: -7.227080931012566 Total: 1.7638; Orientation: 0.0160; Line Search: 1.1760
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 99c13be to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.560e+03, gradient 7.484e-03, dot -0.162; [8288c717-11f8-4d54-b487-51c1e19a4f0d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -7.227080931012566, -6.928123099974883, -4.246419402952333, -2.1263735214982065
LBFGS Accumulation History: 3 points
Removed measurement 99c13be to history. Total: 3
th(0)=-7.227080931012566;dx=-5.595037537247654E-5
Armijo: th(868668.0670208557)=-0.04830058476356669; dx=2.729647766865498E-7 evalInputDelta=-7.178780346248999
New Minimum: -7.227080931012566 > -8.31996006181275
END: th(434334.03351042786)=-8.31996006181275; dx=-8.344720685452447E-6 evalInputDelta=1.0928791308001848
Fitness changed from -7.227080931012566 to -8.31996006181275
Iteration 4 complete. Error: -8.31996006181275 Total: 3.0466; Orientation: 0.5324; Line Search: 1.9379
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 580cf83 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.825e+03, gradient 7.475e-03, dot -0.154; [8288c717-11f8-4d54-b487-51c1e19a4f0d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.31996006181275, -6.928123099974883, -4.246419402952333, -2.1263735214982065
LBFGS Accumulation History: 3 points
Removed measurement 580cf83 to history. Total: 3
th(0)=-8.31996006181275;dx=-5.581137524771994E-5
Armijo: th(935744.3088563365)=1.8263828678288707; dx=-1.7497697066909958E-7 evalInputDelta=-10.146342929641621
New Minimum: -8.31996006181275 > -8.535760423564192
END: th(467872.15442816826)=-8.535760423564192; dx=-1.0467843175845865E-5 evalInputDelta=0.21580036175144102
Fitness changed from -8.31996006181275 to -8.535760423564192
Iteration 5 complete. Error: -8.535760423564192 Total: 2.7103; Orientation: 0.2955; Line Search: 1.8447
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 1f211743 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.784e+03, gradient 7.482e-03, dot -0.145; [8288c717-11f8-4d54-b487-51c1e19a4f0d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.535760423564192, -6.928123099974883, -4.246419402952333, -2.1263735214982065
LBFGS Accumulation History: 3 points
Removed measurement 1f211743 to history. Total: 3
th(0)=-8.535760423564192;dx=-5.5933146720640935E-5
Armijo: th(1008000.0000000003)=4.0316200577644015; dx=-1.1486953447339313E-6 evalInputDelta=-12.567380481328593
Armijo: th(504000.0000000002)=-8.38389542175744; dx=-1.2720499530505756E-5 evalInputDelta=-0.1518650018067511
New Minimum: -8.535760423564192 > -11.407839593283498
END: th(168000.00000000006)=-11.407839593283498; dx=-3.530334796054665E-5 evalInputDelta=2.8720791697193064
Fitness changed from -8.535760423564192 to -11.407839593283498
Iteration 6 complete. Error: -11.407839593283498 Total: 3.3097; Orientation: 0.2939; Line Search: 2.4145
Adding measurement 46328064 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.461e+03, gradient 6.704e-03, dot -0.165; [8288c717-11f8-4d54-b487-51c1e19a4f0d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.407839593283498, -6.928123099974883, -4.246419402952333, -2.1263735214982065
LBFGS Accumulation History: 3 points
Removed measurement 46328064 to history. Total: 3
th(0)=-11.407839593283498;dx=-4.488720327177259E-5
New Minimum: -11.407839593283498 > -12.86553296170258
END: th(361945.0279253566)=-12.86553296170258; dx=-2.0014782788379562E-5 evalInputDelta=1.4576933684190827
Fitness changed from -11.407839593283498 to -12.86553296170258
Iteration 7 complete. Error: -12.86553296170258 Total: 2.0410; Orientation: 0.2988; Line Search: 1.1847
Adding measurement 22beeaf2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.222e+03, gradient 6.838e-03, dot -0.165; [8288c717-11f8-4d54-b487-51c1e19a4f0d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.86553296170258, -6.928123099974883, -4.246419402952333, -2.1263735214982065
LBFGS Accumulation History: 3 points
Removed measurement 22beeaf2 to history. Total: 3
th(0)=-12.86553296170258;dx=-4.6706013112192955E-5
Armijo: th(779786.9240469472)=-6.498967712674826; dx=-6.6367581611792295E-6 evalInputDelta=-6.3665652490277544
New Minimum: -12.86553296170258 > -14.003152819578506
END: th(389893.4620234736)=-14.003152819578506; dx=-1.7969711522018594E-5 evalInputDelta=1.1376198578759258
Fitness changed from -12.86553296170258 to -14.003152819578506
Iteration 8 complete. Error: -14.003152819578506 Total: 3.0772; Orientation: 0.3023; Line Search: 2.2069
Adding measurement 4f2fe875 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.788e+03, gradient 6.994e-03, dot -0.158; [8288c717-11f8-4d54-b487-51c1e19a4f0d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -14.003152819578506, -6.928123099974883, -4.246419402952333, -2.1263735214982065
LBFGS Accumulation History: 3 points
Removed measurement 4f2fe875 to history. Total: 3
th(0)=-14.003152819578506;dx=-4.888705695190742E-5
Armijo: th(840000.0000000003)=-5.4852638381327985; dx=-5.595772947802543E-6 evalInputDelta=-8.517888981445708
New Minimum: -14.003152819578506 > -14.700946088834115
END: th(420000.0000000002)=-14.700946088834115; dx=-1.7294948988477215E-5 evalInputDelta=0.6977932692556088
Fitness changed from -14.003152819578506 to -14.700946088834115
Iteration 9 complete. Error: -14.700946088834115 Total: 2.8840; Orientation: 0.3348; Line Search: 1.8858
Adding measurement 3f097aa9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.151e+04, gradient 7.250e-03, dot -0.154; [8288c717-11f8-4d54-b487-51c1e19a4f0d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -14.700946088834115, -6.928123099974883, -4.246419402952333, -2.1263735214982065
LBFGS Accumulation History: 3 points
Removed measurement 3f097aa9 to history. Total: 3
th(0)=-14.700946088834115;dx=-5.254107083141683E-5
Armijo: th(904862.5698133915)=-3.284399339701231; dx=-4.142195403711509E-6 evalInputDelta=-11.416546749132884
New Minimum: -14.700946088834115 > -14.762293304506901
END: th(452431.2849066958)=-14.762293304506901; dx=-1.618391591609813E-5 evalInputDelta=0.06134721567278589
Fitness changed from -14.700946088834115 to -14.762293304506901
Iteration 10 complete. Error: -14.762293304506901 Total: 2.7088; Orientation: 0.2985; Line Search: 1.7937
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 4129f34 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.283e+04, gradient 7.472e-03, dot -0.141; [8288c717-11f8-4d54-b487-51c1e19a4f0d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -14.762293304506901, -6.928123099974883, -4.246419402952333, -2.1263735214982065
LBFGS Accumulation History: 3 points
Removed measurement 4129f34 to history. Total: 3
th(0)=-14.762293304506901;dx=-5.5787125968069187E-5
Armijo: th(974733.6550586841)=-0.7747918366275179; dx=-3.190096590024609E-6 evalInputDelta=-13.987501467879383
Armijo: th(487366.82752934203)=-14.357325122814828; dx=-1.574872634054087E-5 evalInputDelta=-0.40496818169207316
New Minimum: -14.762293304506901 > -17.277898783739104
END: th(162455.60917644735)=-17.277898783739104; dx=-3.740622632033105E-5 evalInputDelta=2.515605479232203
Fitness changed from -14.762293304506901 to -17.277898783739104
Iteration 11 complete. Error: -17.277898783739104 Total: 3.3291; Orientation: 0.2930; Line Search: 2.4695
Adding measurement 57276fcc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.848e+04, gradient 6.692e-03, dot -0.169; [8288c717-11f8-4d54-b487-51c1e19a4f0d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.277898783739104, -6.928123099974883, -4.246419402952333, -2.1263735214982065
LBFGS Accumulation History: 3 points
Removed measurement 57276fcc to history. Total: 3
th(0)=-17.277898783739104;dx=-4.4742465972676134E-5
New Minimum: -17.277898783739104 > -18.392709491192917
END: th(350000.00000000023)=-18.392709491192917; dx=-2.2788709349506613E-5 evalInputDelta=1.1148107074538132
Fitness changed from -17.277898783739104 to -18.392709491192917
Iteration 12 complete. Error: -18.392709491192917 Total: 2.3839; Orientation: 0.2785; Line Search: 1.1730
Adding measurement 7b5f7481 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.342e+04, gradient 6.773e-03, dot -0.167; [8288c717-11f8-4d54-b487-51c1e19a4f0d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.392709491192917, -6.928123099974883, -4.246419402952333, -2.1263735214982065
LBFGS Accumulation History: 3 points
Removed measurement 7b5f7481 to history. Total: 3
th(0)=-18.392709491192917;dx=-4.582120886993077E-5
Armijo: th(754052.1415111598)=-11.88699040571909; dx=-9.339956103934617E-6 evalInputDelta=-6.505719085473828
New Minimum: -18.392709491192917 > -19.274274992258718
END: th(377026.0707555799)=-19.274274992258718; dx=-2.0650559462288224E-5 evalInputDelta=0.8815655010658006
Fitness changed from -18.392709491192917 to -19.274274992258718
Iteration 13 complete. Error: -19.274274992258718 Total: 2.6993; Orientation: 0.3269; Line Search: 1.8279
Adding measurement 5820e230 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.209e+04, gradient 6.864e-03, dot -0.169; [8288c717-11f8-4d54-b487-51c1e19a4f0d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.274274992258718, -6.928123099974883, -4.246419402952333, -2.1263735214982065
LBFGS Accumulation History: 3 points
Removed measurement 5820e230 to history. Total: 3
th(0)=-19.274274992258718;dx=-4.702823269780743E-5
Armijo: th(812278.0458822369)=-10.741260918528496; dx=-6.682340930538529E-6 evalInputDelta=-8.533014073730222
New Minimum: -19.274274992258718 > -19.602752819622516
END: th(406139.02294111846)=-19.602752819622516; dx=-1.8602177732551752E-5 evalInputDelta=0.32847782736379827
Fitness changed from -19.274274992258718 to -19.602752819622516
Iteration 14 complete. Error: -19.602752819622516 Total: 2.6085; Orientation: 0.2907; Line Search: 1.7849
Adding measurement 50d640ec to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.818e+04, gradient 7.035e-03, dot -0.162; [8288c717-11f8-4d54-b487-51c1e19a4f0d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.602752819622516, -6.928123099974883, -4.246419402952333, -2.1263735214982065
LBFGS Accumulation History: 3 points
Removed measurement 50d640ec to history. Total: 3
th(0)=-19.602752819622516;dx=-4.944145194803407E-5
Armijo: th(875000.0000000007)=-9.285266379937347; dx=-5.373664259911557E-6 evalInputDelta=-10.317486439685169
Armijo: th(437500.00000000035)=-19.587565103541188; dx=-1.7269303228543613E-5 evalInputDelta=-0.015187716081328517
New Minimum: -19.602752819622516 > -21.36324631820282
END: th(145833.33333333346)=-21.36324631820282; dx=-3.457278722990624E-5 evalInputDelta=1.7604934985803027
Fitness changed from -19.602752819622516 to -21.36324631820282
Iteration 15 complete. Error: -21.36324631820282 Total: 3.6193; Orientation: 0.2935; Line Search: 2.7836
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -21.36324631820282 (> -Infinity) after 45.444s (< 5400.000s)
Returns
-21.36324631820282