BasicOptimizer.scala:89 executed in 44.45 seconds (0.744 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 3460736713845
Reset training subject: 3461166889763
Adding measurement 3601108 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-18.181213887498174;dx=-0.00101815701634649
New Minimum: -18.181213887498174 > -18.181329997082315
WOLFE (weak): th(2.154434690031884)=-18.181329997082315; dx=-0.0010177436985632274 evalInputDelta=1.1610958414109973E-4
New Minimum: -18.181329997082315 > -18.18144200797844
WOLFE (weak): th(4.308869380063768)=-18.18144200797844; dx=-0.0010178615294590275 evalInputDelta=2.281204802656589E-4
New Minimum: -18.18144200797844 > -18.18189141061103
WOLFE (weak): th(12.926608140191302)=-18.18189141061103; dx=-0.00101711426335849 evalInputDelta=6.77523112855738E-4
New Minimum: -18.18189141061103 > -18.183904024433485
WOLFE (weak): th(51.70643256076521)=-18.183904024433485; dx=-0.0010164586787510936 evalInputDelta=0.0026901369353105054
New Minimum: -18.183904024433485 > -18.194400562284226
WOLFE (weak): th(258.53216280382605)=-18.194400562284226; dx=-0.0010135442777060573 evalInputDelta=0.013186674786052066
New Minimum: -18.194400562284226 > -18.26266901294133
WOLFE (weak): th(1551.1929768229563)=-18.26266901294133; dx=-9.838466740416745E-4 evalInputDelta=0.08145512544315636
New Minimum: -18.26266901294133 > -18.656640380182843
END: th(10858.350837760694)=-18.656640380182843; dx=-7.748685682460099E-4 evalInputDelta=0.47542649268466874
Fitness changed from -18.181213887498174 to -18.656640380182843
Iteration 1 complete. Error: -18.656640380182843 Total: 5.0819; Orientation: 0.0177; Line Search: 3.7973
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 47b6464c to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-18.656640380182843;dx=-9.601559400335021E-4
New Minimum: -18.656640380182843 > -19.530043691574082
END: th(23393.607721408407)=-19.530043691574082; dx=-5.477024218305373E-4 evalInputDelta=0.8734033113912396
Fitness changed from -18.656640380182843 to -19.530043691574082
Iteration 2 complete. Error: -19.530043691574082 Total: 1.3346; Orientation: 0.0169; Line Search: 0.9081
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 1a5b020c to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-19.530043691574082;dx=-9.347047179864639E-4
New Minimum: -19.530043691574082 > -21.992741077238804
END: th(50400.00000000001)=-21.992741077238804; dx=-3.3672867297198847E-4 evalInputDelta=2.462697385664722
Fitness changed from -19.530043691574082 to -21.992741077238804
Iteration 3 complete. Error: -21.992741077238804 Total: 1.6553; Orientation: 0.0163; Line Search: 1.2003
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 338d2c12 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.504e+03, gradient 3.074e-02, dot -0.348; [facd86e5-1376-4067-9081-e418c0d1dc09 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -21.992741077238804, -19.530043691574082, -18.656640380182843, -18.181213887498174
LBFGS Accumulation History: 3 points
Removed measurement 338d2c12 to history. Total: 3
th(0)=-21.992741077238804;dx=-9.443524061812616E-4
New Minimum: -21.992741077238804 > -24.477762376193972
END: th(108583.50837760697)=-24.477762376193972; dx=-1.6064359564233933E-4 evalInputDelta=2.485021298955168
Fitness changed from -21.992741077238804 to -24.477762376193972
Iteration 4 complete. Error: -24.477762376193972 Total: 1.7447; Orientation: 0.4391; Line Search: 0.8898
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 5872265d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.559e+03, gradient 3.060e-02, dot -0.190; [facd86e5-1376-4067-9081-e418c0d1dc09 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -24.477762376193972, -19.530043691574082, -18.656640380182843, -18.181213887498174
LBFGS Accumulation History: 3 points
Removed measurement 5872265d to history. Total: 3
th(0)=-24.477762376193972;dx=-9.248893227658285E-4
New Minimum: -24.477762376193972 > -29.085869548986487
END: th(233936.07721408413)=-29.085869548986487; dx=-5.8715736288711705E-5 evalInputDelta=4.608107172792515
Fitness changed from -24.477762376193972 to -29.085869548986487
Iteration 5 complete. Error: -29.085869548986487 Total: 1.7333; Orientation: 0.4204; Line Search: 0.9014
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 59f78e37 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.511e+03, gradient 2.493e-02, dot -0.125; [facd86e5-1376-4067-9081-e418c0d1dc09 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -29.085869548986487, -19.530043691574082, -18.656640380182843, -18.181213887498174
LBFGS Accumulation History: 3 points
Removed measurement 59f78e37 to history. Total: 3
th(0)=-29.085869548986487;dx=-6.197578010126378E-4
New Minimum: -29.085869548986487 > -29.644514322474812
END: th(504000.0000000002)=-29.644514322474812; dx=-2.1873060061010363E-5 evalInputDelta=0.5586447734883251
Fitness changed from -29.085869548986487 to -29.644514322474812
Iteration 6 complete. Error: -29.644514322474812 Total: 1.7118; Orientation: 0.4167; Line Search: 0.8909
Adding measurement 759f19ed to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.703e+03, gradient 1.899e-02, dot -0.090; [facd86e5-1376-4067-9081-e418c0d1dc09 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -29.644514322474812, -19.530043691574082, -18.656640380182843, -18.181213887498174
LBFGS Accumulation History: 3 points
Removed measurement 759f19ed to history. Total: 3
th(0)=-29.644514322474812;dx=-3.59433700163581E-4
New Minimum: -29.644514322474812 > -29.7077326813182
END: th(1085835.0837760698)=-29.7077326813182; dx=-8.20357262202384E-6 evalInputDelta=0.06321835884338611
Fitness changed from -29.644514322474812 to -29.7077326813182
Iteration 7 complete. Error: -29.7077326813182 Total: 1.7208; Orientation: 0.4166; Line Search: 0.9103
Adding measurement 6155add1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.437e+03, gradient 1.386e-02, dot -0.060; [facd86e5-1376-4067-9081-e418c0d1dc09 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -29.7077326813182, -19.530043691574082, -18.656640380182843, -18.181213887498174
LBFGS Accumulation History: 3 points
Removed measurement 6155add1 to history. Total: 3
th(0)=-29.7077326813182;dx=-1.9144839269184967E-4
New Minimum: -29.7077326813182 > -30.987912315620516
END: th(2339360.7721408415)=-30.987912315620516; dx=-3.3690555788233136E-6 evalInputDelta=1.2801796343023177
Fitness changed from -29.7077326813182 to -30.987912315620516
Iteration 8 complete. Error: -30.987912315620516 Total: 1.6892; Orientation: 0.4187; Line Search: 0.8959
Adding measurement 47972be7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.399e+03, gradient 9.690e-03, dot -0.041; [facd86e5-1376-4067-9081-e418c0d1dc09 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -30.987912315620516, -19.530043691574082, -18.656640380182843, -18.181213887498174
LBFGS Accumulation History: 3 points
Removed measurement 47972be7 to history. Total: 3
th(0)=-30.987912315620516;dx=-9.3410093516691E-5
Armijo: th(5040000.000000002)=-30.270279769977094; dx=-1.172080918700096E-6 evalInputDelta=-0.717632545643422
New Minimum: -30.987912315620516 > -34.49402248681723
END: th(2520000.000000001)=-34.49402248681723; dx=-7.139603282522927E-6 evalInputDelta=3.5061101711967133
Fitness changed from -30.987912315620516 to -34.49402248681723
Iteration 9 complete. Error: -34.49402248681723 Total: 2.1752; Orientation: 0.4143; Line Search: 1.3613
Adding measurement 15da89db to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.676e+04, gradient 8.944e-03, dot -0.065; [facd86e5-1376-4067-9081-e418c0d1dc09 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -34.49402248681723, -19.
...skipping 433 bytes...
alInputDelta=2.366721033296791
Fitness changed from -34.49402248681723 to -36.86074352011402
Iteration 10 complete. Error: -36.86074352011402 Total: 2.1860; Orientation: 0.4189; Line Search: 1.3691
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 4b35c192 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.251e+04, gradient 8.286e-03, dot -0.077; [facd86e5-1376-4067-9081-e418c0d1dc09 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -36.86074352011402, -19.530043691574082, -18.656640380182843, -18.181213887498174
LBFGS Accumulation History: 3 points
Removed measurement 4b35c192 to history. Total: 3
th(0)=-36.86074352011402;dx=-6.831471630350086E-5
Armijo: th(5848401.930352104)=-36.19211086380336; dx=-1.431440335593626E-6 evalInputDelta=-0.6686326563106633
New Minimum: -36.86074352011402 > -39.85275626525987
END: th(2924200.965176052)=-39.85275626525987; dx=-7.335417938889419E-6 evalInputDelta=2.99201274514585
Fitness changed from -36.86074352011402 to -39.85275626525987
Iteration 11 complete. Error: -39.85275626525987 Total: 2.1580; Orientation: 0.4157; Line Search: 1.3331
Adding measurement 7979595 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.061e+04, gradient 8.086e-03, dot -0.091; [facd86e5-1376-4067-9081-e418c0d1dc09 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -39.85275626525987, -19.530043691574082, -18.656640380182843, -18.181213887498174
LBFGS Accumulation History: 3 points
Removed measurement 7979595 to history. Total: 3
th(0)=-39.85275626525987;dx=-6.512501507163591E-5
Armijo: th(6300000.000000003)=-39.17897176447571; dx=-1.0608150131409516E-6 evalInputDelta=-0.6737845007841585
New Minimum: -39.85275626525987 > -42.54511058333821
END: th(3150000.0000000014)=-42.54511058333821; dx=-6.777156927509686E-6 evalInputDelta=2.692354318078337
Fitness changed from -39.85275626525987 to -42.54511058333821
Iteration 12 complete. Error: -42.54511058333821 Total: 2.1642; Orientation: 0.4170; Line Search: 1.3711
Adding measurement 38a81e3a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.179e+05, gradient 8.031e-03, dot -0.090; [facd86e5-1376-4067-9081-e418c0d1dc09 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -42.54511058333821, -19.530043691574082, -18.656640380182843, -18.181213887498174
LBFGS Accumulation History: 3 points
Removed measurement 38a81e3a to history. Total: 3
th(0)=-42.54511058333821;dx=-6.42060226189705E-5
Armijo: th(6786469.273600437)=-39.10661505648464; dx=-1.0956260454196964E-6 evalInputDelta=-3.438495526853565
New Minimum: -42.54511058333821 > -43.363322468788986
END: th(3393234.6368002184)=-43.363322468788986; dx=-5.639100043345763E-6 evalInputDelta=0.8182118854507792
Fitness changed from -42.54511058333821 to -43.363322468788986
Iteration 13 complete. Error: -43.363322468788986 Total: 2.1898; Orientation: 0.4313; Line Search: 1.3585
Adding measurement 51e56c98 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.486e+05, gradient 7.257e-03, dot -0.089; [facd86e5-1376-4067-9081-e418c0d1dc09 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -43.363322468788986, -19.530043691574082, -18.656640380182843, -18.181213887498174
LBFGS Accumulation History: 3 points
Removed measurement 51e56c98 to history. Total: 3
th(0)=-43.363322468788986;dx=-5.2540720759500106E-5
Armijo: th(7310502.412940131)=-41.07339537732788; dx=-1.3060700066624824E-6 evalInputDelta=-2.2899270914611094
New Minimum: -43.363322468788986 > -45.23562292987018
END: th(3655251.2064700653)=-45.23562292987018; dx=-5.907265524781679E-6 evalInputDelta=1.8723004610811955
Fitness changed from -43.363322468788986 to -45.23562292987018
Iteration 14 complete. Error: -45.23562292987018 Total: 2.1675; Orientation: 0.4140; Line Search: 1.3677
Adding measurement 673b5207 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.408e+05, gradient 7.206e-03, dot -0.094; [facd86e5-1376-4067-9081-e418c0d1dc09 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -45.23562292987018, -19.530043691574082, -18.656640380182843, -18.181213887498174
LBFGS Accumulation History: 3 points
Removed measurement 673b5207 to history. Total: 3
th(0)=-45.23562292987018;dx=-5.177195614198375E-5
Armijo: th(7875000.000000005)=-40.56019539094623; dx=-9.060860449713902E-7 evalInputDelta=-4.675427538923955
Armijo: th(3937500.0000000023)=-45.056249714204846; dx=-5.656257107967479E-6 evalInputDelta=-0.17937321566533626
New Minimum: -45.23562292987018 > -46.826009530489955
END: th(1312500.0000000007)=-46.826009530489955; dx=-2.0180876915616293E-5 evalInputDelta=1.590386600619773
Fitness changed from -45.23562292987018 to -46.826009530489955
Iteration 15 complete. Error: -46.826009530489955 Total: 2.6338; Orientation: 0.4456; Line Search: 1.7847
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Adding measurement 2539d012 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 3.838e+04, gradient 7.215e-03, dot 0.138; [facd86e5-1376-4067-9081-e418c0d1dc09 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-46.826009530489955;dx=-38.26058791256532
Armijo: th(2827695.530666849)=-35.28110952861589; dx=1.159441125202964E-6 evalInputDelta=-11.544900001874062
Armijo: th(1413847.7653334246)=-35.28159805456353; dx=2.3280845905232864E-6 evalInputDelta=-11.544411475926424
Armijo: th(471282.58844447485)=-35.284505136807304; dx=6.972182961103398E-6 evalInputDelta=-11.54150439368265
Armijo: th(117820.64711111871)=-35.28645338136555; dx=2.7804347685544126E-5 evalInputDelta=-11.539556149124408
Armijo: th(23564.12942222374)=-35.27881744912165; dx=1.387969510511243E-4 evalInputDelta=-11.547192081368308
Armijo: th(3927.354903703957)=-35.31232921192424; dx=8.324960792836741E-4 evalInputDelta=-11.513680318565719
Armijo: th(561.0507005291367)=-35.43791266022272; dx=0.005743410441736575 evalInputDelta=-11.388096870267233
Armijo: th(70.1313375661421)=-35.656325126699755; dx=0.04217895492105708 evalInputDelta=-11.1696844037902
Armijo: th(7.792370840682455)=-38.797927757154945; dx=0.24386246613987073 evalInputDelta=-8.02808177333501
Armijo: th(0.7792370840682455)=-44.854816658734755; dx=-4.868205856672739 evalInputDelta=-1.9711928717551999
Armijo: th(0.07083973491529505)=-46.69626883097153; dx=-18.74427415136911 evalInputDelta=-0.12974069951842182
Armijo: th(0.005903311242941254)=-46.813590652309266; dx=-21.459653656225512 evalInputDelta=-0.012418878180689319
Armijo: th(4.5410086484163497E-4)=-46.82503574466121; dx=-21.766631260923877 evalInputDelta=-9.737858287479639E-4
Armijo: th(3.243577606011678E-5)=-46.825935277381795; dx=-21.753264035593524 evalInputDelta=-7.425310815989405E-5
Armijo: th(2.162385070674452E-6)=-46.82600316594244; dx=-21.724548818473043 evalInputDelta=-6.364547516568564E-6
Armijo: th(1.3514906691715326E-7)=-46.826009530489955; dx=-21.748336416076896 evalInputDelta=0.0
Armijo: th(7.94994511277372E-9)=-46.82600528745837; dx=-21.710690736033587 evalInputDelta=-4.2430315829733445E-6
Armijo: th(4.416636173763178E-10)=-46.826009530489955; dx=-21.74035899424095 evalInputDelta=0.0
MIN ALPHA (2.324545354612199E-11): th(0.0)=-46.826009530489955
Fitness changed from -46.826009530489955 to -46.826009530489955
Static Iteration Total: 10.4174; Orientation: 0.4161; Line Search: 9.5945
Iteration 16 failed. Error: -46.826009530489955
Previous Error: 0.0 -> -46.826009530489955
Retrying iteration 16
Reset training subject: 3503500409296
Adding measurement 7a6e000 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-46.826009530489955;dx=-5.191216142961465E-5
END: th(2.1544346900318838E-10)=-46.826009530489955; dx=-4.1332456371809446E-5 evalInputDelta=0.0
Fitness changed from -46.826009530489955 to -46.826009530489955
Static Iteration Total: 1.6853; Orientation: 0.0165; Line Search: 0.9005
Iteration 17 failed. Error: -46.826009530489955
Previous Error: 0.0 -> -46.826009530489955
Optimization terminated 17
Final threshold in iteration 17: -46.826009530489955 (> -Infinity) after 44.449s (< 3600.000s)
Returns
-46.826009530489955