BasicOptimizer.scala:89 executed in 65.04 seconds (1.325 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 151424819404100
Reset training subject: 151425711855700
Adding measurement 5f3e192f to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-5.217246620262092;dx=-1.0828645155716347E-4
New Minimum: -5.217246620262092 > -5.217506179196219
WOLFE (weak): th(2.154434690031884)=-5.217506179196219; dx=-1.0750270644321858E-4 evalInputDelta=2.595589341272131E-4
New Minimum: -5.217506179196219 > -5.217766030918834
WOLFE (weak): th(4.308869380063768)=-5.217766030918834; dx=-1.0823711020495185E-4 evalInputDelta=5.194106567421031E-4
New Minimum: -5.217766030918834 > -5.218805308425437
WOLFE (weak): th(12.926608140191302)=-5.218805308425437; dx=-1.0767707492382831E-4 evalInputDelta=0.0015586881633451455
New Minimum: -5.218805308425437 > -5.22347888760304
WOLFE (weak): th(51.70643256076521)=-5.22347888760304; dx=-1.0825263279536868E-4 evalInputDelta=0.006232267340948283
New Minimum: -5.22347888760304 > -5.248417528128975
WOLFE (weak): th(258.53216280382605)=-5.248417528128975; dx=-1.0816334022715331E-4 evalInputDelta=0.03117090786688337
New Minimum: -5.248417528128975 > -5.404025278432845
WOLFE (weak): th(1551.1929768229563)=-5.404025278432845; dx=-1.0719426421410195E-4 evalInputDelta=0.1867786581707529
New Minimum: -5.404025278432845 > -6.454860476463854
END: th(10858.350837760694)=-6.454860476463854; dx=-9.661321103689867E-5 evalInputDelta=1.2376138562017625
Fitness changed from -5.217246620262092 to -6.454860476463854
Iteration 1 complete. Error: -6.454860476463854 Total: 10.0272; Orientation: 0.0147; Line Search: 7.4517
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 4c4f8609 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-6.454860476463854;dx=-1.0395306916172233E-4
New Minimum: -6.454860476463854 > -8.745508408312432
END: th(23393.607721408407)=-8.745508408312432; dx=-7.206250657504517E-5 evalInputDelta=2.2906479318485777
Fitness changed from -6.454860476463854 to -8.745508408312432
Iteration 2 complete. Error: -8.745508408312432 Total: 2.6109; Orientation: 0.0148; Line Search: 1.6671
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 5eafd1be to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-8.745508408312432;dx=-8.832677045959109E-5
New Minimum: -8.745508408312432 > -12.08151598732901
END: th(50400.00000000001)=-12.08151598732901; dx=-4.12146971001976E-5 evalInputDelta=3.3360075790165773
Fitness changed from -8.745508408312432 to -12.08151598732901
Iteration 3 complete. Error: -12.08151598732901 Total: 2.8333; Orientation: 0.0143; Line Search: 1.7059
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 4a361683 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.598e+03, gradient 9.390e-03, dot -0.363; [f30b760a-9230-4ff1-8472-8b1a9b9375d9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.08151598732901, -8.745508408312432, -6.454860476463854, -5.217246620262092
LBFGS Accumulation History: 3 points
Removed measurement 4a361683 to history. Total: 3
th(0)=-12.08151598732901;dx=-8.711438002751741E-5
New Minimum: -12.08151598732901 > -15.400593823861815
END: th(108583.50837760697)=-15.400593823861815; dx=-1.6476293565935344E-5 evalInputDelta=3.3190778365328057
Fitness changed from -12.08151598732901 to -15.400593823861815
Iteration 4 complete. Error: -15.400593823861815 Total: 2.8597; Orientation: 0.2786; Line Search: 1.6731
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 4482119d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.514e+03, gradient 1.140e-02, dot -0.200; [f30b760a-9230-4ff1-8472-8b1a9b9375d9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -15.400593823861815, -8.745508408312432, -6.454860476463854, -5.217246620262092
LBFGS Accumulation History: 3 points
Removed measurement 4482119d to history. Total: 3
th(0)=-15.400593823861815;dx=-1.2944481973513181E-4
New Minimum: -15.400593823861815 > -16.26570281937149
WOLF (strong): th(233936.07721408413)=-16.26570281937149; dx=2.1603615378293548E-6 evalInputDelta=0.8651089955096758
New Minimum: -16.26570281937149 > -18.396299294690557
END: th(116968.03860704206)=-18.396299294690557; dx=-2.413559762470001E-6 evalInputDelta=2.9957054708287423
Fitness changed from -15.400593823861815 to -18.396299294690557
Iteration 5 complete. Error: -18.396299294690557 Total: 3.8973; Orientation: 0.2764; Line Search: 2.6977
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 16ba3ca3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.214e+03, gradient 1.136e-02, dot -0.179; [f30b760a-9230-4ff1-8472-8b1a9b9375d9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.396299294690557, -8.745508408312432, -6.454860476463854, -5.217246620262092
LBFGS Accumulation History: 3 points
Removed measurement 16ba3ca3 to history. Total: 3
th(0)=-18.396299294690557;dx=-1.2884221708604207E-4
New Minimum: -18.396299294690557 > -19.648149056350352
WOLF (strong): th(252000.0000000001)=-19.648149056350352; dx=6.673380987581019E-6 evalInputDelta=1.2518497616597948
New Minimum: -19.648149056350352 > -22.66426861617775
END: th(126000.00000000004)=-22.66426861617775; dx=-3.5099326393960683E-6 evalInputDelta=4.267969321487193
Fitness changed from -18.396299294690557 to -22.66426861617775
Iteration 6 complete. Error: -22.66426861617775 Total: 3.8258; Orientation: 0.2735; Line Search: 2.5835
Adding measurement 7526c94c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.097e+03, gradient 1.023e-02, dot -0.162; [f30b760a-9230-4ff1-8472-8b1a9b9375d9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -22.66426861617775, -8.745508408312432, -6.454860476463854, -5.217246620262092
LBFGS Accumulation History: 3 points
Removed measurement 7526c94c to history. Total: 3
th(0)=-22.66426861617775;dx=-1.0439795390864044E-4
New Minimum: -22.66426861617775 > -23.54105048994555
WOLF (strong): th(271458.77094401745)=-23.54105048994555; dx=6.4429326208226095E-6 evalInputDelta=0.8767818737678006
New Minimum: -23.54105048994555 > -26.860587494895306
END: th(135729.38547200873)=-26.860587494895306; dx=-2.9601364221641372E-6 evalInputDelta=4.196318878717555
Fitness changed from -22.66426861617775 to -26.860587494895306
Iteration 7 complete. Error: -26.860587494895306 Total: 3.6271; Orientation: 0.2748; Line Search: 2.5467
Adding measurement 28cdeeb6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.044e+04, gradient 9.780e-03, dot -0.214; [f30b760a-9230-4ff1-8472-8b1a9b9375d9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -26.860587494895306, -8.745508408312432, -6.454860476463854, -5.217246620262092
LBFGS Accumulation History: 3 points
Removed measurement 28cdeeb6 to history. Total: 3
th(0)=-26.860587494895306;dx=-9.543578956955564E-5
Armijo: th(292420.0965176052)=-26.211020264666626; dx=9.84595292739433E-6 evalInputDelta=-0.6495672302286799
New Minimum: -26.860587494895306 > -30.495156927551697
WOLF (strong): th(146210.0482588026)=-30.495156927551697; dx=3.027556940352755E-6 evalInputDelta=3.6345694326563915
New Minimum: -30.495156927551697 > -30.63917241855375
END: th(48736.6827529342)=-30.63917241855375; dx=-4.586621070760325E-5 evalInputDelta=3.7785849236584426
Fitness changed from -26.860587494895306 to -30.63917241855375
Iteration 8 complete. Error: -30.63917241855375 Total: 4.7576; Orientation: 0.2763; Line Search: 3.6445
Adding measurement 69e24b0b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.519e+04, gradient 7.532e-03, dot -0.287; [f30b760a-9230-4ff1-8472-8b1a9b9375d9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -30.63917241855375, -8.745508408312432, -6.454860476463854, -5.217246620262092
LBFGS Accumulation History: 3 points
Removed measurement 69e24b0b to history. Total: 3
th(0)=-30.63917241855375;dx=-5.654802945850407E-5
New Minimum: -30.63917241855375 > -33.733292427356666
END: th(105000.00000000004)=-33.733292427356666; dx=-2.4262410740122238E-5 evalInputDelta=3.0941200088029177
Fitness changed from -30.63917241855375 to -33.733292427356666
Iteration 9 complete. Error: -33.733292427356666 Total: 2.8670; Orientation: 0.2967; Line Search: 1.7554
Adding measurement 1b97728a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.297e+04, gradient 8.292e-03, dot -0.271; [f30b760a-9230-4ff1-8472-8b1a9b9375d9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -33.733292427356666, -8.745508408312432, -6.454860476463854, -5.217246620262092
LBFGS Accumulation History: 3 points
Removed measurement 1b97728a to history. Total: 3
th(0)=-33.733292427356666;dx=-6.840951355961065E-5
New Minimum: -33.733292427356666 > -34.29848567184452
END: th(226215.6424533479)=-34.29848567184452; dx=-4.3172889645361875E-6 evalInputDelta=0.5651932444878511
Fitness changed from -33.733292427356666 to -34.29848567184452
Iteration 10 complete. Error: -34.29848567184452 Total: 2.7522; Orientation: 0.2753; Line Search: 1.6672
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 779cea83 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.158e+04, gradient 1.025e-02, dot -0.206; [f30b760a-9230-4ff1-8472-8b1a9b9375d9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -34.29848567184452, -8.745508408312432, -6.454860476463854, -5.217246620262092
LBFGS Accumulation History: 3 points
Removed measurement 779cea83 to history. Total: 3
th(0)=-34.29848567184452;dx=-1.038402805577886E-4
Armijo: th(487366.82752934203)=-27.123066085313795; dx=6.619143285947295E-6 evalInputDelta=-7.175419586530722
New Minimum: -34.29848567184452 > -35.095885867987974
WOLF (strong): th(243683.41376467102)=-35.095885867987974; dx=1.4086710807914339E-5 evalInputDelta=0.7974001961434567
New Minimum: -35.095885867987974 > -39.7573571824382
END: th(81227.80458822368)=-39.7573571824382; dx=-2.198210207115541E-5 evalInputDelta=5.458871510593681
Fitness changed from -34.29848567184452 to -39.7573571824382
Iteration 11 complete. Error: -39.7573571824382 Total: 4.7940; Orientation: 0.2773; Line Search: 3.6186
Adding measurement 2ee38921 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.688e+04, gradient 7.856e-03, dot -0.319; [f30b760a-9230-4ff1-8472-8b1a9b9375d9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -39.7573571824382, -8.745508408312432, -6.454860476463854, -5.217246620262092
LBFGS Accumulation History: 3 points
Removed measurement 2ee38921 to history. Total: 3
th(0)=-39.7573571824382;dx=-6.134820352027418E-5
New Minimum: -39.7573571824382 > -43.08293167666151
END: th(175000.00000000012)=-43.08293167666151; dx=-5.32106745729458E-6 evalInputDelta=3.3255744942233108
Fitness changed from -39.7573571824382 to -43.08293167666151
Iteration 12 complete. Error: -43.08293167666151 Total: 2.7745; Orientation: 0.2744; Line Search: 1.6953
Adding measurement 792ff991 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.576e+04, gradient 9.453e-03, dot -0.245; [f30b760a-9230-4ff1-8472-8b1a9b9375d9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -43.08293167666151, -8.745508408312432, -6.454860476463854, -5.217246620262092
LBFGS Accumulation History: 3 points
Removed measurement 792ff991 to history. Total: 3
th(0)=-43.08293167666151;dx=-8.858203899968093E-5
Armijo: th(377026.0707555799)=-36.97667243774944; dx=1.0212308794779372E-5 evalInputDelta=-6.106259238912067
New Minimum: -43.08293167666151 > -43.94523027643036
WOLF (strong): th(188513.03537778996)=-43.94523027643036; dx=1.3407362976931554E-5 evalInputDelta=0.8622985997688488
New Minimum: -43.94523027643036 > -46.92185219807949
END: th(62837.67845926332)=-46.92185219807949; dx=-2.6776132010152093E-5 evalInputDelta=3.8389205214179825
Fitness changed from -43.08293167666151 to -46.92185219807949
Iteration 13 complete. Error: -46.92185219807949 Total: 4.5484; Orientation: 0.2689; Line Search: 3.4560
Adding measurement 672b25b1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.984e+04, gradient 6.556e-03, dot -0.327; [f30b760a-9230-4ff1-8472-8b1a9b9375d9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -46.92185219807949, -8.745508408312432, -6.454860476463854, -5.217246620262092
LBFGS Accumulation History: 3 points
Removed measurement 672b25b1 to history. Total: 3
th(0)=-46.92185219807949;dx=-4.263136356493829E-5
New Minimum: -46.92185219807949 > -49.89367751738509
END: th(135379.67431370614)=-49.89367751738509; dx=-2.1131513679191405E-5 evalInputDelta=2.9718253193056015
Fitness changed from -46.92185219807949 to -49.89367751738509
Iteration 14 complete. Error: -49.89367751738509 Total: 2.8911; Orientation: 0.2633; Line Search: 1.7247
Adding measurement 4df93ec to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.092e+04, gradient 7.148e-03, dot 0.520; [f30b760a-9230-4ff1-8472-8b1a9b9375d9 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-49.89367751738509;dx=-40.60692623469886
New Minimum: -49.89367751738509 > -55.742874249087876
Armijo: th(291666.66666666686)=-55.742874249087876; dx=7.77764674041005E-5 evalInputDelta=5.849196731702783
Armijo: th(145833.33333333343)=-55.73643302308031; dx=1.5685591432453944E-4 evalInputDelta=5.842755505695216
WOLF (strong): th(48611.111111111146)=-55.729767575945665; dx=4.6931704693042484E-4 evalInputDelta=5.836090058560572
WOLF (strong): th(12152.777777777786)=-55.738337813429126; dx=0.0018896559744475735 evalInputDelta=5.844660296044033
New Minimum: -55.742874249087876 > -55.85992471739549
WOLF (strong): th(2430.5555555555575)=-55.85992471739549; dx=0.009462806594173992 evalInputDelta=5.966247200010393
New Minimum: -55.85992471739549 > -56.3243222134989
WOLF (strong): th(405.0925925925929)=-56.3243222134989; dx=0.05714000008708017 evalInputDelta=6.430644696113809
New Minimum: -56.3243222134989 > -59.922304107655314
WOLF (strong): th(57.870370370370416)=-59.922304107655314; dx=0.41013423526056614 evalInputDelta=10.028626590270221
New Minimum: -59.922304107655314 > -82.18228061211914
WOLF (strong): th(7.233796296296302)=-82.18228061211914; dx=0.9768107616219011 evalInputDelta=32.28860309473405
END: th(0.8037551440329225)=-71.91515248157745; dx=-17.327514862237933 evalInputDelta=22.021474964192358
Fitness changed from -49.89367751738509 to -82.18228061211914
Iteration 15 complete. Error: -82.18228061211914 Total: 9.8704; Orientation: 0.2675; Line Search: 8.8141
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -82.18228061211914 (> -Infinity) after 65.041s (< 5400.000s)
Returns
-82.18228061211914