BasicOptimizer.scala:89 executed in 157.53 seconds (2.025 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 2769398681487
Reset training subject: 2770516639909
Adding measurement 61a7dec2 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-4.989797416952617;dx=-4.295593589959272E-4
New Minimum: -4.989797416952617 > -4.99065643941647
WOLFE (weak): th(2.154434690031884)=-4.99065643941647; dx=-4.2998262376859654E-4 evalInputDelta=8.590224638531652E-4
New Minimum: -4.99065643941647 > -4.99151447001526
WOLFE (weak): th(4.308869380063768)=-4.99151447001526; dx=-4.3064838207256375E-4 evalInputDelta=0.0017170530626433589
New Minimum: -4.99151447001526 > -4.994947227113701
WOLFE (weak): th(12.926608140191302)=-4.994947227113701; dx=-4.255833353109434E-4 evalInputDelta=0.0051498101610842895
New Minimum: -4.994947227113701 > -5.010390124461874
WOLFE (weak): th(51.70643256076521)=-5.010390124461874; dx=-4.234889875569032E-4 evalInputDelta=0.020592707509257124
New Minimum: -5.010390124461874 > -5.092374156572341
WOLFE (weak): th(258.53216280382605)=-5.092374156572341; dx=-4.105647508582834E-4 evalInputDelta=0.10257673961972458
New Minimum: -5.092374156572341 > -5.616647579693195
END: th(1551.1929768229563)=-5.616647579693195; dx=-3.37242160406053E-4 evalInputDelta=0.6268501627405785
Fitness changed from -4.989797416952617 to -5.616647579693195
Iteration 1 complete. Error: -5.616647579693195 Total: 11.0214; Orientation: 0.0183; Line Search: 7.7627
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 70cac4f5 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-5.616647579693195;dx=-3.6026685898140156E-4
New Minimum: -5.616647579693195 > -6.633338582559375
END: th(3341.943960201201)=-6.633338582559375; dx=-2.515685799623504E-4 evalInputDelta=1.01669100286618
Fitness changed from -5.616647579693195 to -6.633338582559375
Iteration 2 complete. Error: -6.633338582559375 Total: 3.2921; Orientation: 0.0191; Line Search: 2.1919
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 7657793a to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-6.633338582559375;dx=-4.698003717843219E-4
New Minimum: -6.633338582559375 > -8.286034049013903
END: th(7200.000000000001)=-8.286034049013903; dx=-1.6075035637299638E-4 evalInputDelta=1.652695466454528
Fitness changed from -6.633338582559375 to -8.286034049013903
Iteration 3 complete. Error: -8.286034049013903 Total: 3.3384; Orientation: 0.0188; Line Search: 2.2419
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 3825a4c8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.652e+03, gradient 3.259e-02, dot -0.357; [dab07250-1614-45e3-abb2-8d756d15bd8c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.286034049013903, -6.633338582559375, -5.616647579693195, -4.989797416952617
LBFGS Accumulation History: 3 points
Removed measurement 3825a4c8 to history. Total: 3
th(0)=-8.286034049013903;dx=-0.0010595201779714251
New Minimum: -8.286034049013903 > -12.029826924715554
END: th(15511.929768229566)=-12.029826924715554; dx=-4.4605982336282966E-5 evalInputDelta=3.7437928757016508
Fitness changed from -8.286034049013903 to -12.029826924715554
Iteration 4 complete. Error: -12.029826924715554 Total: 3.7650; Orientation: 0.4638; Line Search: 2.2217
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement dc48ae4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.543e+03, gradient 5.820e-02, dot -0.112; [dab07250-1614-45e3-abb2-8d756d15bd8c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.029826924715554, -6.633338582559375, -5.616647579693195, -4.989797416952617
LBFGS Accumulation History: 3 points
Removed measurement dc48ae4 to history. Total: 3
th(0)=-12.029826924715554;dx=-0.0033810361534470563
New Minimum: -12.029826924715554 > -15.117175417033897
WOLF (strong): th(33419.43960201202)=-15.117175417033897; dx=8.04376290034418E-5 evalInputDelta=3.087348492318343
New Minimum: -15.117175417033897 > -17.096233008782306
WOLF (strong): th(16709.71980100601)=-17.096233008782306; dx=1.0910653416303144E-4 evalInputDelta=5.066406084066752
New Minimum: -17.096233008782306 > -19.161138069770818
END: th(5569.906600335336)=-19.161138069770818; dx=-2.928156957912597E-4 evalInputDelta=7.131311145055264
Fitness changed from -12.029826924715554 to -19.161138069770818
Iteration 5 complete. Error: -19.161138069770818 Total: 5.9980; Orientation: 0.4602; Line Search: 4.4610
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 2aa69f2a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.859e+04, gradient 5.747e-02, dot -0.147; [dab07250-1614-45e3-abb2-8d756d15bd8c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.161138069770818, -6.633338582559375, -5.616647579693195, -4.989797416952617
LBFGS Accumulation History: 3 points
Removed measurement 2aa69f2a to history. Total: 3
th(0)=-19.161138069770818;dx=-0.0032999294234506904
New Minimum: -19.161138069770818 > -23.48232064329528
WOLF (strong): th(12000.000000000004)=-23.48232064329528; dx=2.4565133838032294E-4 evalInputDelta=4.321182573524464
New Minimum: -23.48232064329528 > -25.790028734351665
END: th(6000.000000000002)=-25.790028734351665; dx=-6.601047261940445E-5 evalInputDelta=6.628890664580847
Fitness changed from -19.161138069770818 to -25.790028734351665
Iteration 6 complete. Error: -25.790028734351665 Total: 4.8398; Orientation: 0.4553; Line Search: 3.3107
Adding measurement 3ce40ce to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.306e+04, gradient 7.589e-02, dot -0.107; [dab07250-1614-45e3-abb2-8d756d15bd8c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -25.790028734351665, -6.633338582559375, -5.616647579693195, -4.989797416952617
LBFGS Accumulation History: 3 points
Removed measurement 3ce40ce to history. Total: 3
th(0)=-25.790028734351665;dx=-0.0057564033386321885
New Minimum: -25.790028734351665 > -29.12749430193973
WOLF (strong): th(12926.608140191307)=-29.12749430193973; dx=3.085618454532884E-5 evalInputDelta=3.3374655675880653
New Minimum: -29.12749430193973 > -31.3045250835335
END: th(6463.3040700956535)=-31.3045250835335; dx=-3.7196753058167746E-5 evalInputDelta=5.514496349181833
Fitness changed from -25.790028734351665 to -31.3045250835335
Iteration 7 complete. Error: -31.3045250835335 Total: 4.8355; Orientation: 0.4585; Line Search: 3.3264
Adding measurement 134a69e3 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 3.563e+02, gradient 1.062e-01, dot 0.392; [dab07250-1614-45e3-abb2-8d756d15bd8c = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-31.3045250835335;dx=-14.85026150438749
Armijo: th(13924.766500838343)=-27.982983514192853; dx=-5.497660524318869E-5 evalInputDelta=-3.3215415693406456
Armijo: th(6962.383250419171)=-28.29282988651926; dx=-1.5093949080486604E-4 evalInputDelta=-3.0116951970142374
Armijo: th(2320.7944168063905)=-28.583798219463965; dx=-7.470311705211281E-4 evalInputDelta=-2.7207268640695332
Armijo: th(580.1986042015976)=-26.58743207469232; dx=-0.006604553277767844 evalInputDelta=-4.717093008841179
Armijo: th(116.03972084031952)=-13.848637566550357; dx=-0.019155762489032002 evalInputDelta=-17.45588751698314
Armijo: th(19.339953473386586)=-6.6586918984032355; dx=-0.07870269935647224 evalInputDelta=-24.645833185130265
Armijo: th(2.7628504961980838)=-28.73983417750995; dx=2.586450363675683 evalInputDelta=-2.5646909060235465
New Minimum: -31.3045250835335 > -34.99939183849345
END: th(0.34535631202476047)=-34.99939183849345; dx=-10.971690348958116 evalInputDelta=3.6948667549599534
Fitness changed from -31.3045250835335 to -34.99939183849345
Iteration 8 complete. Error: -34.99939183849345 Total: 12.6964; Orientation: 0.4430; Line Search: 11.2026
Adding measurement 6ce62228 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 2.636e+04, gradient 1.00
...skipping 6224 bytes...
50149011 evalInputDelta=12.962562007660111
Fitness changed from -60.889664158518514 to -73.85222616617862
Iteration 13 complete. Error: -73.85222616617862 Total: 3.7200; Orientation: 0.4482; Line Search: 2.2157
Adding measurement 178bf81 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.055e+04, gradient 8.888e-02, dot -0.050; [dab07250-1614-45e3-abb2-8d756d15bd8c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -73.85222616617862, -6.633338582559375, -5.616647579693195, -4.989797416952617
LBFGS Accumulation History: 3 points
Removed measurement 178bf81 to history. Total: 3
th(0)=-73.85222616617862;dx=-0.007892617661748357
New Minimum: -73.85222616617862 > -94.29042467773232
END: th(4144.275744297127)=-94.29042467773232; dx=-0.003852906546877741 evalInputDelta=20.438198511553693
Fitness changed from -73.85222616617862 to -94.29042467773232
Iteration 14 complete. Error: -94.29042467773232 Total: 3.7237; Orientation: 0.4511; Line Search: 2.2218
Adding measurement 563886e4 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 3.302e+03, gradient 1.019e-01, dot 0.166; [dab07250-1614-45e3-abb2-8d756d15bd8c = 1.000/1.000e+00]
th(0)=-94.29042467773232;dx=-55.839911676631644
Armijo: th(8928.571428571435)=-34.28365484869815; dx=3.6705912454483493E-4 evalInputDelta=-60.00676982903417
Armijo: th(4464.285714285717)=-34.246704177787315; dx=7.867706600205471E-4 evalInputDelta=-60.043720499945
Armijo: th(1488.0952380952392)=-34.13806862586002; dx=0.002427362895024027 evalInputDelta=-60.152356051872296
Armijo: th(372.0238095238098)=-33.69196472482228; dx=0.002966685870463782 evalInputDelta=-60.59845995291004
Armijo: th(74.40476190476195)=-33.83854596208552; dx=0.027649388034919477 evalInputDelta=-60.4518787156468
Armijo: th(12.40079365079366)=-37.03895605442375; dx=0.08945068316334576 evalInputDelta=-57.25146862330857
Armijo: th(1.7715419501133798)=-86.33737189276067; dx=3.5378097760130918 evalInputDelta=-7.953052784971646
New Minimum: -94.29042467773232 > -101.55435316178873
END: th(0.22144274376417247)=-101.55435316178873; dx=-33.4441119206781 evalInputDelta=7.263928484056407
Fitness changed from -94.29042467773232 to -101.55435316178873
Iteration 15 complete. Error: -101.55435316178873 Total: 11.6061; Orientation: 0.4518; Line Search: 10.0906
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Adding measurement 58baead9 to history. Total: 4
Accepted: LBFGS Orientation magnitude: 3.387e+03, gradient 1.090e-01, dot 0.046; [dab07250-1614-45e3-abb2-8d756d15bd8c = 1.000/1.000e+00]
th(0)=-101.55435316178873;dx=-17.09269656093443
Armijo: th(0.4770839290213748)=-97.25262008891823; dx=1.6158391835654295 evalInputDelta=-4.301733072870491
Armijo: th(0.2385419645106874)=-100.6852811405825; dx=-4.159468315362285 evalInputDelta=-0.8690720212062217
New Minimum: -101.55435316178873 > -101.60957369739747
END: th(0.07951398817022913)=-101.60957369739747; dx=-10.14261999632142 evalInputDelta=0.05522053560873985
Fitness changed from -101.55435316178873 to -101.60957369739747
Iteration 16 complete. Error: -101.60957369739747 Total: 6.2261; Orientation: 0.5666; Line Search: 4.4650
Adding measurement 629c609c to history. Total: 5
Accepted: LBFGS Orientation magnitude: 1.175e+03, gradient 1.128e-01, dot 0.033; [dab07250-1614-45e3-abb2-8d756d15bd8c = 1.000/1.000e+00]
th(0)=-101.60957369739747;dx=-4.378601241934347
Armijo: th(0.17130769445672647)=-101.37178217462937; dx=-1.6886988966647491 evalInputDelta=-0.23779152276809157
Armijo: th(0.08565384722836324)=-101.52881373099967; dx=-2.61434485195674 evalInputDelta=-0.08075996639779248
Armijo: th(0.028551282409454412)=-101.59442050155513; dx=-2.8873145074882736 evalInputDelta=-0.015153195842330547
Armijo: th(0.007137820602363603)=-101.60648485802376; dx=-3.120322565724104 evalInputDelta=-0.0030888393737029674
Armijo: th(0.0014275641204727206)=-101.60906951181609; dx=-3.237155574657567 evalInputDelta=-5.041855813772145E-4
Armijo: th(2.379273534121201E-4)=-101.60948878210633; dx=-3.2374380922308266 evalInputDelta=-8.491529113996421E-5
Armijo: th(3.3989621916017155E-5)=-101.60955777578577; dx=-3.2353140703261065 evalInputDelta=-1.5921611691283033E-5
Armijo: th(4.248702739502144E-6)=-101.60957369739747; dx=-3.2401111176137727 evalInputDelta=0.0
Armijo: th(4.7207808216690494E-7)=-101.60957369739747; dx=-3.241283482622701 evalInputDelta=0.0
Armijo: th(4.7207808216690496E-8)=-101.60956839019384; dx=-3.236590433613744 evalInputDelta=-5.307203622351153E-6
Armijo: th(4.291618928790045E-9)=-101.60957369739747; dx=-3.2402514586381983 evalInputDelta=0.0
Armijo: th(3.5763491073250373E-10)=-101.60956839019384; dx=-3.2272476222009603 evalInputDelta=-5.307203622351153E-6
MIN ALPHA (2.7510377748654132E-11): th(0.0)=-101.60957369739747
Fitness changed from -101.60957369739747 to -101.60957369739747
Static Iteration Total: 18.4501; Orientation: 1.7586; Line Search: 15.6270
Iteration 17 failed. Error: -101.60957369739747
Previous Error: 0.0 -> -101.60957369739747
Retrying iteration 17
Reset training subject: 2894654560121
Adding measurement 66f600f4 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-101.60957369739747;dx=-0.012686815273634399
WOLFE (weak): th(2.1544346900318838E-10)=-101.60957369739747; dx=-0.01264922048458942 evalInputDelta=0.0
WOLFE (weak): th(4.3088693800637676E-10)=-101.60957369739747; dx=-0.01264531086741125 evalInputDelta=0.0
WOLFE (weak): th(1.2926608140191302E-9)=-101.60957369739747; dx=-0.012647898676124645 evalInputDelta=0.0
WOLFE (weak): th(5.170643256076521E-9)=-101.60957369739747; dx=-0.012650740218072765 evalInputDelta=0.0
WOLFE (weak): th(2.5853216280382606E-8)=-101.60957369739747; dx=-0.012646482044176188 evalInputDelta=0.0
WOLFE (weak): th(1.5511929768229565E-7)=-101.60957369739747; dx=-0.012647804576080933 evalInputDelta=0.0
Armijo: th(1.0858350837760695E-6)=-101.60957369739747; dx=-0.01264974050369403 evalInputDelta=0.0
Armijo: th(6.204771907291826E-7)=-101.60957369739747; dx=-0.012647940238187223 evalInputDelta=0.0
WOLFE (weak): th(3.877982442057391E-7)=-101.60957369739747; dx=-0.012647256581139237 evalInputDelta=0.0
WOLFE (weak): th(5.041377174674609E-7)=-101.60957369739747; dx=-0.012647013030096962 evalInputDelta=0.0
Armijo: th(5.623074540983217E-7)=-101.60957369739747; dx=-0.012647774805984262 evalInputDelta=0.0
WOLFE (weak): th(5.332225857828913E-7)=-101.60957369739747; dx=-0.012649035222333987 evalInputDelta=0.0
WOLFE (weak): th(5.477650199406065E-7)=-101.60957369739747; dx=-0.012646427372527594 evalInputDelta=0.0
WOLFE (weak): th(5.550362370194641E-7)=-101.60957369739747; dx=-0.012645667540428736 evalInputDelta=0.0
WOLFE (weak): th(5.58671845558893E-7)=-101.60957369739747; dx=-0.012646100059997894 evalInputDelta=0.0
Armijo: th(5.604896498286073E-7)=-101.60957369739747; dx=-0.012650205211616541 evalInputDelta=0.0
WOLFE (weak): th(5.595807476937501E-7)=-101.60957369739747; dx=-0.012646314542200585 evalInputDelta=0.0
WOLFE (weak): th(5.600351987611787E-7)=-101.60957369739747; dx=-0.01265084391897844 evalInputDelta=0.0
Armijo: th(5.602624242948931E-7)=-101.60957369739747; dx=-0.012645861822799922 evalInputDelta=0.0
Armijo: th(5.601488115280359E-7)=-101.60957369739747; dx=-0.01264835863018493 evalInputDelta=0.0
Armijo: th(5.600920051446072E-7)=-101.60957369739747; dx=-0.01264798211047317 evalInputDelta=0.0
WOLFE (weak): th(5.600636019528929E-7)=-101.60957369739747; dx=-0.012651351239961368 evalInputDelta=0.0
Armijo: th(5.600778035487501E-7)=-101.60957369739747; dx=-0.01264905926390874 evalInputDelta=0.0
Armijo: th(5.600707027508215E-7)=-101.60957369739747; dx=-0.012646348470452429 evalInputDelta=0.0
Armijo: th(5.600671523518573E-7)=-101.60957369739747; dx=-0.012650834227086193 evalInputDelta=0.0
mu ~= nu (5.600636019528929E-7): th(0.0)=-101.60957369739747
Fitness changed from -101.60957369739747 to -101.60957369739747
Static Iteration Total: 32.2712; Orientation: 0.0183; Line Search: 30.1466
Iteration 18 failed. Error: -101.60957369739747
Previous Error: 0.0 -> -101.60957369739747
Optimization terminated 18
Final threshold in iteration 18: -101.60957369739747 (> -Infinity) after 157.527s (< 3600.000s)
Returns
-101.60957369739747