BasicOptimizer.scala:89 executed in 124.93 seconds (3.564 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 4885132629674
Reset training subject: 4886262323323
Adding measurement 2f24934d to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.6491731005219608;dx=-1.03505336936943E-5
New Minimum: -0.6491731005219608 > -0.6492026372423692
WOLFE (weak): th(2.154434690031884)=-0.6492026372423692; dx=-1.0344705772233629E-5 evalInputDelta=2.9536720408396278E-5
New Minimum: -0.6492026372423692 > -0.6492318812778356
WOLFE (weak): th(4.308869380063768)=-0.6492318812778356; dx=-1.0325075623288366E-5 evalInputDelta=5.8780755874843216E-5
New Minimum: -0.6492318812778356 > -0.6493498273482006
WOLFE (weak): th(12.926608140191302)=-0.6493498273482006; dx=-1.0339371488465444E-5 evalInputDelta=1.7672682623981473E-4
New Minimum: -0.6493498273482006 > -0.6498805926266664
WOLFE (weak): th(51.70643256076521)=-0.6498805926266664; dx=-1.0335538614515739E-5 evalInputDelta=7.074921047056648E-4
New Minimum: -0.6498805926266664 > -0.6527306030497336
WOLFE (weak): th(258.53216280382605)=-0.6527306030497336; dx=-1.043134899903154E-5 evalInputDelta=0.003557502527772871
New Minimum: -0.6527306030497336 > -0.6705075669141285
WOLFE (weak): th(1551.1929768229563)=-0.6705075669141285; dx=-1.0262008847883688E-5 evalInputDelta=0.02133446639216774
New Minimum: -0.6705075669141285 > -0.799674991479776
END: th(10858.350837760694)=-0.799674991479776; dx=-9.241607042493263E-6 evalInputDelta=0.1505018909578153
Fitness changed from -0.6491731005219608 to -0.799674991479776
Iteration 1 complete. Error: -0.799674991479776 Total: 12.5371; Orientation: 0.0196; Line Search: 9.1937
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 2672f0c to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-0.799674991479776;dx=-1.248143105938573E-5
New Minimum: -0.799674991479776 > -1.1905093859971556
END: th(23393.607721408407)=-1.1905093859971556; dx=-1.000876128412028E-5 evalInputDelta=0.39083439451737956
Fitness changed from -0.799674991479776 to -1.1905093859971556
Iteration 2 complete. Error: -1.1905093859971556 Total: 3.4272; Orientation: 0.0197; Line Search: 2.2845
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 1990141e to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-1.1905093859971556;dx=-3.038719365348062E-5
New Minimum: -1.1905093859971556 > -2.2259613974109445
END: th(50400.00000000001)=-2.2259613974109445; dx=-5.733138911246674E-6 evalInputDelta=1.0354520114137888
Fitness changed from -1.1905093859971556 to -2.2259613974109445
Iteration 3 complete. Error: -2.2259613974109445 Total: 3.4332; Orientation: 0.0213; Line Search: 2.2904
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 1ac0da4f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.798e+03, gradient 1.017e-02, dot -0.022; [16114d8f-75b2-4ce2-a3b9-9a6d2f1dbe0b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.2259613974109445, -1.1905093859971556, -0.799674991479776, -0.6491731005219608
LBFGS Accumulation History: 3 points
Removed measurement 1ac0da4f to history. Total: 3
th(0)=-2.2259613974109445;dx=-1.0318067125578434E-4
New Minimum: -2.2259613974109445 > -2.825858202882419
WOLF (strong): th(108583.50837760697)=-2.825858202882419; dx=4.247480192877005E-6 evalInputDelta=0.5998968054714746
New Minimum: -2.825858202882419 > -3.3492302332851063
WOLF (strong): th(54291.75418880348)=-3.3492302332851063; dx=1.123883323752095E-6 evalInputDelta=1.1232688358741618
New Minimum: -3.3492302332851063 > -3.372436990999398
END: th(18097.251396267828)=-3.372436990999398; dx=-2.884660305688167E-5 evalInputDelta=1.1464755935884536
Fitness changed from -2.2259613974109445 to -3.372436990999398
Iteration 4 complete. Error: -3.372436990999398 Total: 8.6028; Orientation: 0.4738; Line Search: 7.0222
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 1aee2453 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.339e+05, gradient 1.216e-02, dot -0.146; [16114d8f-75b2-4ce2-a3b9-9a6d2f1dbe0b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.372436990999398, -1.1905093859971556, -0.799674991479776, -0.6491731005219608
LBFGS Accumulation History: 3 points
Removed measurement 1aee2453 to history. Total: 3
th(0)=-3.372436990999398;dx=-1.4753438945165723E-4
New Minimum: -3.372436990999398 > -5.121910573702642
END: th(38989.346202347355)=-5.121910573702642; dx=-1.081778190587414E-5 evalInputDelta=1.7494735827032435
Fitness changed from -3.372436990999398 to -5.121910573702642
Iteration 5 complete. Error: -5.121910573702642 Total: 3.9282; Orientation: 0.4925; Line Search: 2.2786
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 72eda4bf to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.404e+04, gradient 1.658e-02, dot -0.055; [16114d8f-75b2-4ce2-a3b9-9a6d2f1dbe0b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.121910573702642, -1.1905093859971556, -0.799674991479776, -0.6491731005219608
LBFGS Accumulation History: 3 points
Removed measurement 72eda4bf to history. Total: 3
th(0)=-5.121910573702642;dx=-2.746291267381293E-4
New Minimum: -5.121910573702642 > -5.472705510299875
WOLF (strong): th(84000.00000000003)=-5.472705510299875; dx=5.3584676296211325E-6 evalInputDelta=0.35079493659723315
New Minimum: -5.472705510299875 > -6.342916440341746
WOLF (strong): th(42000.000000000015)=-6.342916440341746; dx=8.095019862230743E-6 evalInputDelta=1.2210058666391044
New Minimum: -6.342916440341746 > -6.941624975988658
END: th(14000.000000000005)=-6.941624975988658; dx=-4.196846809126654E-5 evalInputDelta=1.8197144022860163
Fitness changed from -5.121910573702642 to -6.941624975988658
Iteration 6 complete. Error: -6.941624975988658 Total: 6.1654; Orientation: 0.4691; Line Search: 4.5818
Adding measurement 12e79c56 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.129e+06, gradient 1.416e-02, dot -0.066; [16114d8f-75b2-4ce2-a3b9-9a6d2f1dbe0b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.941624975988658, -1.1905093859971556, -0.799674991479776, -0.6491731005219608
LBFGS Accumulation History: 3 points
Removed measurement 12e79c56 to history. Total: 3
th(0)=-6.941624975988658;dx=-1.9989499015141116E-4
New Minimum: -6.941624975988658 > -8.591484605664585
END: th(30162.085660446384)=-8.591484605664585; dx=-8.367909833490015E-6 evalInputDelta=1.6498596296759267
Fitness changed from -6.941624975988658 to -8.591484605664585
Iteration 7 complete. Error: -8.591484605664585 Total: 3.8297; Orientation: 0.4702; Line Search: 2.2911
Adding measurement 54518f9d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.466e+04, gradient 2.053e-02, dot -0.014; [16114d8f-75b2-4ce2-a3b9-9a6d2f1dbe0b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.591484605664585, -1.1905093859971556, -0.799674991479776, -0.6491731005219608
LBFGS Accumulation History: 3 points
Removed measurement 54518f9d to history. Total: 3
th(0)=-8.591484605664585;dx=-4.2096099801701734E-4
New Minimum: -8.591484605664585 > -8.818456050716412
WOLF (strong): th(64982.24367057894)=-8.818456050716412; dx=1.872713625220107E-5 evalInputDelta=0.2269714450518272
New Minimum: -8.818456050716412 > -10.280266582212418
WOLF (strong): th(32491.12183528947)=-10.280266582212418; dx=2.0590888270570452E-5 evalInputDelta=1.6887819765478334
New Minimum: -10.280266582212418 > -10.542252676443132
END: th(10830.37394509649)=-10.542252676443132; dx=-7.158217078618266E-5 evalInputDelta=1.9507680707785475
Fitness changed from -8.591484605664585 to -10.542252676443132
Iteration 8 complete. Error: -10.542252676443132 Total: 6.2416; Orientation: 0.5564; Line Search: 4.5897
Adding measurement 308a83fc to history. Total: 3
Accepted: LBFGS Orientation magnitude: 2.064e+04, gradient 1.597e-02, dot 0.077; [16114d8f-75b2-4ce2-a
...skipping 19 bytes...
1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-10.542252676443132;dx=-25.148598147907002
Armijo: th(23333.333333333347)=-0.11138449990096888; dx=-3.1474046179543626E-7 evalInputDelta=-10.430868176542162
Armijo: th(11666.666666666673)=-0.11465047898617686; dx=-1.2759617707932567E-6 evalInputDelta=-10.427602197456956
Armijo: th(3888.888888888891)=-0.1244448622114491; dx=-4.6213078243591595E-6 evalInputDelta=-10.417807814231683
Armijo: th(972.2222222222227)=-0.11534935700570335; dx=1.087057836800921E-5 evalInputDelta=-10.426903319437429
Armijo: th(194.44444444444454)=-0.15010210788792996; dx=6.251157356626847E-4 evalInputDelta=-10.392150568555202
Armijo: th(32.407407407407426)=-0.3130145667282995; dx=0.01111951173481579 evalInputDelta=-10.229238109714833
Armijo: th(4.629629629629632)=-0.41004378999302754; dx=0.08945180204848444 evalInputDelta=-10.132208886450105
Armijo: th(0.578703703703704)=-0.544330760093269; dx=1.0724647889193486 evalInputDelta=-9.997921916349863
Armijo: th(0.06430041152263379)=-6.939738695981214; dx=47.45765808158637 evalInputDelta=-3.602513980461918
New Minimum: -10.542252676443132 > -10.543970939621872
WOLF (strong): th(0.006430041152263378)=-10.543970939621872; dx=9.186878035954408 evalInputDelta=0.0017182631787395763
New Minimum: -10.543970939621872 > -10.554083671389343
END: th(5.845491956603071E-4)=-10.554083671389343; dx=-20.524876165903216 evalInputDelta=0.011830994946210538
Fitness changed from -10.542252676443132 to -10.554083671389343
Iteration 9 complete. Error: -10.554083671389343 Total: 15.4043; Orientation: 0.4589; Line Search: 13.8483
Adding measurement 5a23de9f to history. Total: 4
Accepted: LBFGS Orientation magnitude: 1.146e+04, gradient 1.589e-02, dot 0.055; [16114d8f-75b2-4ce2-a3b9-9a6d2f1dbe0b = 1.000/1.000e+00]
th(0)=-10.554083671389343;dx=-9.988274678349459
New Minimum: -10.554083671389343 > -10.562023619933672
END: th(0.0012593730651608008)=-10.562023619933672; dx=-7.952922607210368 evalInputDelta=0.007939948544329667
Fitness changed from -10.554083671389343 to -10.562023619933672
Iteration 10 complete. Error: -10.562023619933672 Total: 4.0298; Orientation: 0.6598; Line Search: 2.2857
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 23c564e7 to history. Total: 5
Accepted: LBFGS Orientation magnitude: 9.629e+03, gradient 1.595e-02, dot 0.041; [16114d8f-75b2-4ce2-a3b9-9a6d2f1dbe0b = 1.000/1.000e+00]
th(0)=-10.562023619933672;dx=-6.26429923304276
New Minimum: -10.562023619933672 > -10.567738407261437
END: th(0.0027132370192742136)=-10.567738407261437; dx=-3.3233963864969596 evalInputDelta=0.005714787327764981
Fitness changed from -10.562023619933672 to -10.567738407261437
Iteration 11 complete. Error: -10.567738407261437 Total: 4.0842; Orientation: 0.6784; Line Search: 2.2881
Adding measurement 3880b14a to history. Total: 6
Accepted: LBFGS Orientation magnitude: 6.603e+03, gradient 1.607e-02, dot 0.013; [16114d8f-75b2-4ce2-a3b9-9a6d2f1dbe0b = 1.000/1.000e+00]
th(0)=-10.567738407261437;dx=-1.341113507273819
Armijo: th(0.005845491956603072)=-10.553769963799018; dx=2.1460539772564013 evalInputDelta=-0.013968443462418989
Armijo: th(0.002922745978301536)=-10.563601152829486; dx=0.5217841968788737 evalInputDelta=-0.004137254431951121
Armijo: th(9.742486594338454E-4)=-10.566850025206376; dx=-0.6699834503194166 evalInputDelta=-8.883820550611432E-4
Armijo: th(2.4356216485846134E-4)=-10.56759292621019; dx=-1.1211675893017337 evalInputDelta=-1.4548105124667643E-4
Armijo: th(4.871243297169227E-5)=-10.56771549699253; dx=-1.306636000771075 evalInputDelta=-2.2910268906883857E-5
Armijo: th(8.118738828615378E-6)=-10.567734397967964; dx=-1.3120767812356406 evalInputDelta=-4.0092934732172125E-6
Armijo: th(1.1598198326593396E-6)=-10.56773783450532; dx=-1.345724654031232 evalInputDelta=-5.727561180890461E-7
Armijo: th(1.4497747908241745E-7)=-10.56773783450532; dx=-1.3307364437471314 evalInputDelta=-5.727561180890461E-7
Armijo: th(1.610860878693527E-8)=-10.56773783450532; dx=-1.3317661629035287 evalInputDelta=-5.727561180890461E-7
Armijo: th(1.6108608786935271E-9)=-10.567738407261437; dx=-1.333290572931757 evalInputDelta=0.0
Armijo: th(1.4644189806304792E-10)=-10.56773783450532; dx=-1.3718772580859153 evalInputDelta=-5.727561180890461E-7
MIN ALPHA (1.2203491505253994E-11): th(0.0)=-10.567738407261437
Fitness changed from -10.567738407261437 to -10.567738407261437
Static Iteration Total: 16.6641; Orientation: 0.8009; Line Search: 14.7671
Iteration 12 failed. Error: -10.567738407261437
Previous Error: 0.0 -> -10.567738407261437
Retrying iteration 12
Reset training subject: 4973480622030
Adding measurement 7d673652 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-10.567738407261437;dx=-2.5791706057417987E-4
WOLFE (weak): th(2.1544346900318838E-10)=-10.567738407261437; dx=-2.579561314766123E-4 evalInputDelta=0.0
WOLFE (weak): th(4.3088693800637676E-10)=-10.567738407261437; dx=-2.5789217527980355E-4 evalInputDelta=0.0
WOLFE (weak): th(1.2926608140191302E-9)=-10.567738407261437; dx=-2.579120146378969E-4 evalInputDelta=0.0
WOLFE (weak): th(5.170643256076521E-9)=-10.567738407261437; dx=-2.5790164248926757E-4 evalInputDelta=0.0
WOLFE (weak): th(2.5853216280382606E-8)=-10.567738407261437; dx=-2.5786389824080974E-4 evalInputDelta=0.0
WOLFE (weak): th(1.5511929768229565E-7)=-10.567738407261437; dx=-2.5795138313857607E-4 evalInputDelta=0.0
WOLFE (weak): th(1.0858350837760695E-6)=-10.567738407261437; dx=-2.5788661917711207E-4 evalInputDelta=0.0
Armijo: th(8.686680670208556E-6)=-10.567738407261437; dx=-2.578726344038473E-4 evalInputDelta=0.0
Armijo: th(4.886257876992313E-6)=-10.567738407261437; dx=-2.578908573737451E-4 evalInputDelta=0.0
Armijo: th(2.986046480384191E-6)=-10.56773783450532; dx=-2.5780082204748594E-4 evalInputDelta=-5.727561180890461E-7
WOLFE (weak): th(2.0359407820801304E-6)=-10.567738407261437; dx=-2.5787611267227807E-4 evalInputDelta=0.0
WOLFE (weak): th(2.5109936312321608E-6)=-10.567738407261437; dx=-2.5798456645337856E-4 evalInputDelta=0.0
Armijo: th(2.7485200558081758E-6)=-10.56773783450532; dx=-2.5793046189976124E-4 evalInputDelta=-5.727561180890461E-7
WOLFE (weak): th(2.629756843520168E-6)=-10.567738407261437; dx=-2.578453811304031E-4 evalInputDelta=0.0
WOLFE (weak): th(2.689138449664172E-6)=-10.567738407261437; dx=-2.57929922359005E-4 evalInputDelta=0.0
Armijo: th(2.718829252736174E-6)=-10.56773783450532; dx=-2.5793207264896735E-4 evalInputDelta=-5.727561180890461E-7
Armijo: th(2.703983851200173E-6)=-10.56773783450532; dx=-2.5789160999047035E-4 evalInputDelta=-5.727561180890461E-7
Armijo: th(2.6965611504321728E-6)=-10.56773783450532; dx=-2.578978142724509E-4 evalInputDelta=-5.727561180890461E-7
Armijo: th(2.6928498000481726E-6)=-10.56773783450532; dx=-2.578582247050556E-4 evalInputDelta=-5.727561180890461E-7
Armijo: th(2.690994124856172E-6)=-10.56773783450532; dx=-2.5796296147472664E-4 evalInputDelta=-5.727561180890461E-7
Armijo: th(2.690066287260172E-6)=-10.56773783450532; dx=-2.5783771173089E-4 evalInputDelta=-5.727561180890461E-7
WOLFE (weak): th(2.6896023684621723E-6)=-10.567738407261437; dx=-2.578651343102269E-4 evalInputDelta=0.0
WOLFE (weak): th(2.6898343278611722E-6)=-10.567738407261437; dx=-2.579493160659586E-4 evalInputDelta=0.0
Armijo: th(2.689950307560672E-6)=-10.56773783450532; dx=-2.5789633453835635E-4 evalInputDelta=-5.727561180890461E-7
Armijo: th(2.6898923177109222E-6)=-10.56773783450532; dx=-2.578504104089754E-4 evalInputDelta=-5.727561180890461E-7
WOLFE (weak): th(2.6898633227860474E-6)=-10.567738407261437; dx=-2.578486922364062E-4 evalInputDelta=0.0
Armijo: th(2.6898778202484846E-6)=-10.56773783450532; dx=-2.5791779001712553E-4 evalInputDelta=-5.727561180890461E-7
mu ~= nu (2.6898633227860474E-6): th(0.0)=-10.567738407261437
Fitness changed from -10.567738407261437 to -10.567738407261437
Static Iteration Total: 36.5845; Orientation: 0.0199; Line Search: 34.3894
Iteration 13 failed. Error: -10.567738407261437
Previous Error: 0.0 -> -10.567738407261437
Optimization terminated 13
Final threshold in iteration 13: -10.567738407261437 (> -Infinity) after 124.933s (< 3600.000s)
Returns
-10.567738407261437