BasicOptimizer.scala:89 executed in 8.12 seconds (0.542 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 169631342098500
Reset training subject: 169631468523100
Adding measurement 5ecf2e1f to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-3.2887833420132666;dx=-4.3466369145655565E-5
New Minimum: -3.2887833420132666 > -3.288981325480863
WOLFE (weak): th(2.154434690031884)=-3.288981325480863; dx=-4.5332594151859974E-5 evalInputDelta=1.9798346759625218E-4
New Minimum: -3.288981325480863 > -3.2891794255804507
WOLFE (weak): th(4.308869380063768)=-3.2891794255804507; dx=-4.485424471777888E-5 evalInputDelta=3.960835671841245E-4
New Minimum: -3.2891794255804507 > -3.2899708070691744
WOLFE (weak): th(12.926608140191302)=-3.2899708070691744; dx=-4.454582801655571E-5 evalInputDelta=0.0011874650559078326
New Minimum: -3.2899708070691744 > -3.293534099670344
WOLFE (weak): th(51.70643256076521)=-3.293534099670344; dx=-4.512378088722753E-5 evalInputDelta=0.004750757657077553
New Minimum: -3.293534099670344 > -3.3125843847358727
WOLFE (weak): th(258.53216280382605)=-3.3125843847358727; dx=-4.717509309466695E-5 evalInputDelta=0.02380104272260608
New Minimum: -3.3125843847358727 > -3.434413151029237
WOLFE (weak): th(1551.1929768229563)=-3.434413151029237; dx=-4.516920742336146E-5 evalInputDelta=0.14562980901597022
New Minimum: -3.434413151029237 > -4.351062398275763
WOLFE (weak): th(10858.350837760694)=-4.351062398275763; dx=-4.5417946865733283E-5 evalInputDelta=1.0622790562624966
New Minimum: -4.351062398275763 > -5.297619958479041
END: th(86866.80670208555)=-5.297619958479041; dx=-1.389926766431138E-6 evalInputDelta=2.0088366164657745
Fitness changed from -3.2887833420132666 to -5.297619958479041
Iteration 1 complete. Error: -5.297619958479041 Total: 1.3344; Orientation: 0.0015; Line Search: 0.9891
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 39b58322 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-5.297619958479041;dx=-5.8036653981940594E-5
New Minimum: -5.297619958479041 > -6.684443708562532
END: th(187148.86177126726)=-6.684443708562532; dx=-4.961053980885543E-6 evalInputDelta=1.386823750083491
Fitness changed from -5.297619958479041 to -6.684443708562532
Iteration 2 complete. Error: -6.684443708562532 Total: 0.3530; Orientation: 0.0015; Line Search: 0.2329
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 156bbd01 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-6.684443708562532;dx=-2.0886196797366573E-5
New Minimum: -6.684443708562532 > -12.663552313980675
END: th(403200.00000000006)=-12.663552313980675; dx=-4.076455701610076E-6 evalInputDelta=5.979108605418143
Fitness changed from -6.684443708562532 to -12.663552313980675
Iteration 3 complete. Error: -12.663552313980675 Total: 0.3499; Orientation: 0.0015; Line Search: 0.2264
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 11c1be85 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.836e+03, gradient 4.811e-03, dot -0.365; [8960b819-1df9-4b47-8020-fa022879dc9f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.663552313980675, -6.684443708562532, -5.297619958479041, -3.2887833420132666
LBFGS Accumulation History: 3 points
Removed measurement 11c1be85 to history. Total: 3
th(0)=-12.663552313980675;dx=-2.238400022005824E-5
New Minimum: -12.663552313980675 > -19.723955381335784
END: th(868668.0670208557)=-19.723955381335784; dx=-3.116482565072074E-6 evalInputDelta=7.060403067355109
Fitness changed from -12.663552313980675 to -19.723955381335784
Iteration 4 complete. Error: -19.723955381335784 Total: 0.3817; Orientation: 0.0330; Line Search: 0.2312
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 115b51ad to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.484e+03, gradient 5.028e-03, dot -0.317; [8960b819-1df9-4b47-8020-fa022879dc9f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.723955381335784, -6.684443708562532, -5.297619958479041, -3.2887833420132666
LBFGS Accumulation History: 3 points
Removed measurement 115b51ad to history. Total: 3
th(0)=-19.723955381335784;dx=-2.326669193540811E-5
New Minimum: -19.723955381335784 > -27.29939168156972
END: th(1871488.617712673)=-27.29939168156972; dx=-4.6674215037204045E-7 evalInputDelta=7.575436300233935
Fitness changed from -19.723955381335784 to -27.29939168156972
Iteration 5 complete. Error: -27.29939168156972 Total: 0.3715; Orientation: 0.0265; Line Search: 0.2263
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 251266a8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.730e+04, gradient 4.447e-03, dot -0.256; [8960b819-1df9-4b47-8020-fa022879dc9f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -27.29939168156972, -6.684443708562532, -5.297619958479041, -3.2887833420132666
LBFGS Accumulation History: 3 points
Removed measurement 251266a8 to history. Total: 3
th(0)=-27.29939168156972;dx=-1.8883640738250663E-5
New Minimum: -27.29939168156972 > -38.416004339546774
WOLF (strong): th(4032000.0000000014)=-38.416004339546774; dx=6.016467349967234E-7 evalInputDelta=11.116612657977054
New Minimum: -38.416004339546774 > -41.70451977200553
END: th(2016000.0000000007)=-41.70451977200553; dx=-8.28273504238401E-7 evalInputDelta=14.405128090435813
Fitness changed from -27.29939168156972 to -41.70451977200553
Iteration 6 complete. Error: -41.70451977200553 Total: 0.4835; Orientation: 0.0276; Line Search: 0.3388
Adding measurement 19c312f3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.088e+06, gradient 4.431e-03, dot -0.311; [8960b819-1df9-4b47-8020-fa022879dc9f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -41.70451977200553, -6.684443708562532, -5.297619958479041, -3.2887833420132666
LBFGS Accumulation History: 3 points
Removed measurement 19c312f3 to history. Total: 3
th(0)=-41.70451977200553;dx=-1.902221707100349E-5
Armijo: th(4343340.335104279)=-36.035553584634144; dx=1.7570837614240208E-6 evalInputDelta=-5.668966187371389
New Minimum: -41.70451977200553 > -49.939638087639494
WOLF (strong): th(2171670.1675521396)=-49.939638087639494; dx=1.3478811280182792E-6 evalInputDelta=8.235118315633962
New Minimum: -49.939638087639494 > -51.90524051549776
END: th(723890.0558507132)=-51.90524051549776; dx=-5.7428326185950185E-6 evalInputDelta=10.200720743492226
Fitness changed from -41.70451977200553 to -51.90524051549776
Iteration 7 complete. Error: -51.90524051549776 Total: 0.5792; Orientation: 0.0277; Line Search: 0.4436
Adding measurement 32d46f54 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 3.100e+04, gradient 4.647e-03, dot 0.420; [8960b819-1df9-4b47-8020-fa022879dc9f = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-51.90524051549776;dx=-60.63064292235628
Armijo: th(1559573.8480938943)=-49.349043763874654; dx=4.03908135410571E-6 evalInputDelta=-2.556196751623105
Armijo: th(779786.9240469472)=-49.349043763874654; dx=8.056145424537464E-6 evalInputDelta=-2.556196751623105
Armijo: th(259928.97468231572)=-49.349043763874654; dx=2.4216432864126236E-5 evalInputDelta=-2.556196751623105
Armijo: th(64982.24367057893)=-49.349043763874654; dx=9.908322559944388E-5 evalInputDelta=-2.556196751623105
Armijo: th(12996.448734115786)=-49.349043763874654; dx=4.8256583538474517E-4 evalInputDelta=-2.556196751623105
Armijo: th(2166.0747890192974)=-49.34183353257018; dx=0.002879126923638655 evalInputDelta=-2.5634069829275816
Armijo: th(309.4392555741853)=-49.35091469829044; dx=0.02025101807140469 evalInputDelta=-2.554325817207321
Armijo: th(38.679906946773166)=-49.468699993329835; dx=0.1688811538670683 evalInputDelta=-2.436540522167924
Armijo: th(4.297767438530352)=-50.57708809618343; dx=1.2402005012224957 evalInputDelta=-1.3281524193143284
New Minimum: -51.90524051549776 > -54.94756894812183
END: th(0.4297767438530352)=-54.94756894812183; dx=-0.053173987369150244 evalInputDelta=3.0423284326240676
Fitness changed from -51.90524051549776 to -54.94756894812183
Iteration 8 complete. Error: -54.94756894812183 Total: 1.3638; Orientation: 0.0263; Line Search: 1.2287
Adding measurement 7f159421 to history. Total: 4
Accepted: LBFGS Orientation magnitude: 2.194e+04, gradient 4.612e-03, dot 0.348; [8960b819-1df9-4b47-8020-fa022879dc9f = 1.000/1.000e+00]
th(0)=-54.94756894812183;dx=-35.025568175857856
Armijo: th(0.9259259259259263)=-53.93195306265135; dx=0.26703768285537954 evalInputDelta=-1.0156158854704742
Armijo: th(0.46296296296296313)=-54.49059812482934; dx=-0.043075557726645036 evalInputDelta=-0.4569708232924867
Armijo: th(0.15432098765432103)=-54.81039587829599; dx=-0.25628893024760413 evalInputDelta=-0.13717306982583466
Armijo: th(0.03858024691358026)=-54.91839263455303; dx=-0.425129978307519 evalInputDelta=-0.02917631356879724
Armijo: th(0.0077160493827160516)=-54.94188544238065; dx=-0.45662125680973603 evalInputDelta=-0.005683505741174599
Armijo: th(0.0012860082304526753)=-54.94658004066988; dx=-0.40049555564179024 evalInputDelta=-9.889074519477958E-4
Armijo: th(1.8371546149323933E-4)=-54.94743696349507; dx=-0.39368423470846536 evalInputDelta=-1.3198462675489964E-4
Armijo: th(2.2964432686654917E-5)=-54.94755318878008; dx=-0.4400349026002809 evalInputDelta=-1.5759341749799205E-5
Armijo: th(2.5516036318505464E-6)=-54.94756697820436; dx=-0.484705731386323 evalInputDelta=-1.969917470034943E-6
Armijo: th(2.5516036318505463E-7)=-54.94756894812183; dx=-0.4237506718567691 evalInputDelta=0.0
Armijo: th(2.3196396653186786E-8)=-54.947563038369196; dx=-0.4585483207973056 evalInputDelta=-5.909752630373077E-6
Armijo: th(1.9330330544322323E-9)=-54.947563038369196; dx=-0.43042444032111604 evalInputDelta=-5.909752630373077E-6
Armijo: th(1.4869485034094095E-10)=-54.94756894812183; dx=-0.4262057891753525 evalInputDelta=0.0
MIN ALPHA (1.0621060738638638E-11): th(0.0)=-54.94756894812183
Fitness changed from -54.94756894812183 to -54.94756894812183
Static Iteration Total: 2.4567; Orientation: 0.0366; Line Search: 2.3014
Iteration 9 failed. Error: -54.94756894812183
Previous Error: 0.0 -> -54.94756894812183
Retrying iteration 9
Reset training subject: 169639015953300
Adding measurement 4f89c88f to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-54.94756894812183;dx=-2.032505190689216E-5
END: th(2.1544346900318838E-10)=-54.94756894812183; dx=-1.0670489102761065E-5 evalInputDelta=0.0
Fitness changed from -54.94756894812183 to -54.94756894812183
Static Iteration Total: 0.4467; Orientation: 0.0015; Line Search: 0.2336
Iteration 10 failed. Error: -54.94756894812183
Previous Error: 0.0 -> -54.94756894812183
Optimization terminated 10
Final threshold in iteration 10: -54.94756894812183 (> -Infinity) after 8.121s (< 5400.000s)
Returns
-54.94756894812183