BasicOptimizer.scala:89 executed in 42.72 seconds (0.941 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 214588496153700
Reset training subject: 214589261255300
Adding measurement 6297aada to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-2.2000657965760904;dx=-2.272457407067278E-5
New Minimum: -2.2000657965760904 > -2.2000951991223228
WOLFE (weak): th(2.154434690031884)=-2.2000951991223228; dx=-2.2735974158317467E-5 evalInputDelta=2.9402546232404347E-5
New Minimum: -2.2000951991223228 > -2.2001248001824147
WOLFE (weak): th(4.308869380063768)=-2.2001248001824147; dx=-2.2723015361432435E-5 evalInputDelta=5.900360632438151E-5
New Minimum: -2.2001248001824147 > -2.200243132026888
WOLFE (weak): th(12.926608140191302)=-2.200243132026888; dx=-2.2722613987122812E-5 evalInputDelta=1.7733545079767765E-4
New Minimum: -2.200243132026888 > -2.2007739467937397
WOLFE (weak): th(51.70643256076521)=-2.2007739467937397; dx=-2.270534226441107E-5 evalInputDelta=7.081502176493615E-4
New Minimum: -2.2007739467937397 > -2.203587660354361
WOLFE (weak): th(258.53216280382605)=-2.203587660354361; dx=-2.264603080000166E-5 evalInputDelta=0.003521863778270795
New Minimum: -2.203587660354361 > -2.220880912370997
WOLFE (weak): th(1551.1929768229563)=-2.220880912370997; dx=-2.218868045924473E-5 evalInputDelta=0.020815115794906802
New Minimum: -2.220880912370997 > -2.329136293892191
END: th(10858.350837760694)=-2.329136293892191; dx=-1.8385451574183284E-5 evalInputDelta=0.12907049731610076
Fitness changed from -2.2000657965760904 to -2.329136293892191
Iteration 1 complete. Error: -2.329136293892191 Total: 6.8518; Orientation: 0.0161; Line Search: 4.9571
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 2b9ab4ea to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-2.329136293892191;dx=-2.1200021205308756E-5
New Minimum: -2.329136293892191 > -2.5571085755680842
END: th(23393.607721408407)=-2.5571085755680842; dx=-1.5663586340597544E-5 evalInputDelta=0.22797228167589312
Fitness changed from -2.329136293892191 to -2.5571085755680842
Iteration 2 complete. Error: -2.5571085755680842 Total: 1.8347; Orientation: 0.0166; Line Search: 1.2471
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 3ab24018 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-2.5571085755680842;dx=-2.419666664975791E-5
New Minimum: -2.5571085755680842 > -3.019174135963426
END: th(50400.00000000001)=-3.019174135963426; dx=-1.2969674320112322E-5 evalInputDelta=0.46206556039534163
Fitness changed from -2.5571085755680842 to -3.019174135963426
Iteration 3 complete. Error: -3.019174135963426 Total: 1.8047; Orientation: 0.0165; Line Search: 1.1979
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 694a58d6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.872e+03, gradient 5.751e-03, dot -0.505; [4dadb465-ee51-4b2a-9902-778a0baeeaf0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.019174135963426, -2.5571085755680842, -2.329136293892191, -2.2000657965760904
LBFGS Accumulation History: 3 points
Removed measurement 694a58d6 to history. Total: 3
th(0)=-3.019174135963426;dx=-3.306441347956346E-5
New Minimum: -3.019174135963426 > -3.518403322172093
END: th(108583.50837760697)=-3.518403322172093; dx=-8.383915190781126E-6 evalInputDelta=0.4992291862086673
Fitness changed from -3.019174135963426 to -3.518403322172093
Iteration 4 complete. Error: -3.518403322172093 Total: 2.5381; Orientation: 0.3119; Line Search: 1.6580
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 26ad71a0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.138e+03, gradient 5.810e-03, dot -0.311; [4dadb465-ee51-4b2a-9902-778a0baeeaf0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.518403322172093, -2.5571085755680842, -2.329136293892191, -2.2000657965760904
LBFGS Accumulation History: 3 points
Removed measurement 26ad71a0 to history. Total: 3
th(0)=-3.518403322172093;dx=-3.373989411543533E-5
New Minimum: -3.518403322172093 > -3.6279662840487266
END: th(233936.07721408413)=-3.6279662840487266; dx=-2.4213413029707647E-6 evalInputDelta=0.10956296187663339
Fitness changed from -3.518403322172093 to -3.6279662840487266
Iteration 5 complete. Error: -3.6279662840487266 Total: 2.1107; Orientation: 0.3038; Line Search: 1.2109
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 7a6791e7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.242e+03, gradient 5.792e-03, dot -0.148; [4dadb465-ee51-4b2a-9902-778a0baeeaf0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.6279662840487266, -2.5571085755680842, -2.329136293892191, -2.2000657965760904
LBFGS Accumulation History: 3 points
Removed measurement 7a6791e7 to history. Total: 3
th(0)=-3.6279662840487266;dx=-3.353649448503671E-5
Armijo: th(504000.0000000002)=-1.6174910737354145; dx=1.2533770707744968E-7 evalInputDelta=-2.010475210313312
New Minimum: -3.6279662840487266 > -3.874080259239987
END: th(252000.0000000001)=-3.874080259239987; dx=-3.4235478201879724E-6 evalInputDelta=0.24611397519126044
Fitness changed from -3.6279662840487266 to -3.874080259239987
Iteration 6 complete. Error: -3.874080259239987 Total: 2.7745; Orientation: 0.3133; Line Search: 1.8683
Adding measurement 5c4b19a6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.311e+03, gradient 5.835e-03, dot -0.126; [4dadb465-ee51-4b2a-9902-778a0baeeaf0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.874080259239987, -2.5571085755680842, -2.329136293892191, -2.2000657965760904
LBFGS Accumulation History: 3 points
Removed measurement 5c4b19a6 to history. Total: 3
th(0)=-3.874080259239987;dx=-3.402921349672714E-5
Armijo: th(542917.5418880349)=-2.0040787285295636; dx=4.600972317759087E-7 evalInputDelta=-1.8700015307104234
New Minimum: -3.874080259239987 > -4.63021913660219
END: th(271458.77094401745)=-4.63021913660219; dx=-5.28040170478887E-6 evalInputDelta=0.7561388773622033
Fitness changed from -3.874080259239987 to -4.63021913660219
Iteration 7 complete. Error: -4.63021913660219 Total: 2.6721; Orientation: 0.3079; Line Search: 1.8269
Adding measurement 4390416e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.434e+03, gradient 5.766e-03, dot -0.128; [4dadb465-ee51-4b2a-9902-778a0baeeaf0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.63021913660219, -2.5571085755680842, -2.329136293892191, -2.2000657965760904
LBFGS Accumulation History: 3 points
Removed measurement 4390416e to history. Total: 3
th(0)=-4.63021913660219;dx=-3.322957245658503E-5
Armijo: th(584840.1930352104)=-2.3243789509804618; dx=4.6390773448298874E-7 evalInputDelta=-2.3058401856217285
New Minimum: -4.63021913660219 > -5.168387389214482
END: th(292420.0965176052)=-5.168387389214482; dx=-4.190625589414563E-6 evalInputDelta=0.5381682526122917
Fitness changed from -4.63021913660219 to -5.168387389214482
Iteration 8 complete. Error: -5.168387389214482 Total: 2.6873; Orientation: 0.3068; Line Search: 1.8301
Adding measurement 2130710 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.479e+03, gradient 5.470e-03, dot -0.131; [4dadb465-ee51-4b2a-9902-778a0baeeaf0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.168387389214482, -2.5571085755680842, -2.329136293892191, -2.2000657965760904
LBFGS Accumulation History: 3 points
Removed measurement 2130710 to history. Total: 3
th(0)=-5.168387389214482;dx=-2.9885012474523994E-5
Armijo: th(630000.0000000002)=-2.400232359614339; dx=5.686303731820977E-7 evalInputDelta=-2.768155029600143
New Minimum: -5.168387389214482 > -5.714529183166984
END: th(315000.0000000001)=-5.714529183166984; dx=-4.630217012872805E-6 evalInputDelta=0.5461417939525024
Fitness changed from -5.168387389214482 to -5.714529183166984
Iteration 9 complete. Error: -5.714529183166984 Total: 2.9315; Orientation: 0.2909; Line Search: 1.8310
Adding measurement 5066389b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.621e+03, gradient 5.390e-03, dot -0.141; [4dadb465-ee51-4b2a-9902-778a0baeeaf0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.714529183166984, -2.5571085755680842, -2.329136293892191, -2.2000657965760904
LBFGS Accumulation History: 3 points
Removed measurement 5066389b to history. Total: 3
th(0)=-5.714529183166984;dx=-2.902805655922956E-5
Armijo: th(678646.9273600436)=-2.3672575127496494; dx=9.931786597423122E-7 evalInputDelta=-3.347271670417335
New Minimum: -5.714529183166984 > -6.052776993571841
END: th(339323.4636800218)=-6.052776993571841; dx=-4.09038119117728E-6 evalInputDelta=0.33824781040485696
Fitness changed from -5.714529183166984 to -6.052776993571841
Iteration 10 complete. Error: -6.052776993571841 Total: 2.7316; Orientation: 0.3030; Line Search: 1.8563
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement b7466b4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.796e+03, gradient 5.476e-03, dot -0.131; [4dadb465-ee51-4b2a-9902-778a0baeeaf0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.052776993571841, -2.5571085755680842, -2.329136293892191, -2.2000657965760904
LBFGS Accumulation History: 3 points
Removed measurement b7466b4 to history. Total: 3
th(0)=-6.052776993571841;dx=-2.9963945729448893E-5
Armijo: th(731050.241294013)=-2.2887359725116685; dx=1.1967572876629541E-6 evalInputDelta=-3.764041021060173
New Minimum: -6.052776993571841 > -6.536891960185246
END: th(365525.1206470065)=-6.536891960185246; dx=-3.6393528270318738E-6 evalInputDelta=0.48411496661340436
Fitness changed from -6.052776993571841 to -6.536891960185246
Iteration 11 complete. Error: -6.536891960185246 Total: 2.7156; Orientation: 0.3026; Line Search: 1.8366
Adding measurement 2b579c75 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.020e+04, gradient 5.102e-03, dot -0.136; [4dadb465-ee51-4b2a-9902-778a0baeeaf0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.536891960185246, -2.5571085755680842, -2.329136293892191, -2.2000657965760904
LBFGS Accumulation History: 3 points
Removed measurement 2b579c75 to history. Total: 3
th(0)=-6.536891960185246;dx=-2.601410663056569E-5
Armijo: th(787500.0000000003)=-2.5520430090121575; dx=1.3531204421990062E-6 evalInputDelta=-3.9848489511730882
New Minimum: -6.536891960185246 > -6.972591625776916
END: th(393750.0000000002)=-6.972591625776916; dx=-3.7410635158405023E-6 evalInputDelta=0.4356996655916703
Fitness changed from -6.536891960185246 to -6.972591625776916
Iteration 12 complete. Error: -6.972591625776916 Total: 2.6777; Orientation: 0.3060; Line Search: 1.8097
Adding measurement 6a6309ba to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.268e+04, gradient 5.126e-03, dot -0.132; [4dadb465-ee51-4b2a-9902-778a0baeeaf0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.972591625776916, -2.5571085755680842, -2.329136293892191, -2.2000657965760904
LBFGS Accumulation History: 3 points
Removed measurement 6a6309ba to history. Total: 3
th(0)=-6.972591625776916;dx=-2.6259071401747755E-5
Armijo: th(848308.6592000546)=-1.630043234463443; dx=1.6176557660514365E-6 evalInputDelta=-5.342548391313473
Armijo: th(424154.3296000273)=-6.795878471178581; dx=-2.989087950636755E-6 evalInputDelta=-0.17671315459833536
New Minimum: -6.972591625776916 > -8.125221446982708
END: th(141384.77653334243)=-8.125221446982708; dx=-1.4235686027900628E-5 evalInputDelta=1.1526298212057924
Fitness changed from -6.972591625776916 to -8.125221446982708
Iteration 13 complete. Error: -8.125221446982708 Total: 3.6094; Orientation: 0.2861; Line Search: 2.4183
Adding measurement 192d7187 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.700e+04, gradient 4.269e-03, dot -0.184; [4dadb465-ee51-4b2a-9902-778a0baeeaf0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.125221446982708, -2.5571085755680842, -2.329136293892191, -2.2000657965760904
LBFGS Accumulation History: 3 points
Removed measurement 192d7187 to history. Total: 3
th(0)=-8.125221446982708;dx=-1.8186085399430544E-5
New Minimum: -8.125221446982708 > -8.787967843880475
END: th(304604.26720583875)=-8.787967843880475; dx=-7.482055640655281E-6 evalInputDelta=0.6627463968977665
Fitness changed from -8.125221446982708 to -8.787967843880475
Iteration 14 complete. Error: -8.787967843880475 Total: 2.0636; Orientation: 0.3098; Line Search: 1.2160
Adding measurement 42bdf736 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.763e+04, gradient 4.252e-03, dot -0.183; [4dadb465-ee51-4b2a-9902-778a0baeeaf0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.787967843880475, -2.5571085755680842, -2.329136293892191, -2.2000657965760904
LBFGS Accumulation History: 3 points
Removed measurement 42bdf736 to history. Total: 3
th(0)=-8.787967843880475;dx=-1.8047677495739572E-5
Armijo: th(656250.0000000003)=-6.906753401585213; dx=-1.5529849307157365E-6 evalInputDelta=-1.8812144422952617
New Minimum: -8.787967843880475 > -9.306957376572495
END: th(328125.0000000002)=-9.306957376572495; dx=-5.92708825461066E-6 evalInputDelta=0.5189895326920198
Fitness changed from -8.787967843880475 to -9.306957376572495
Iteration 15 complete. Error: -9.306957376572495 Total: 2.6983; Orientation: 0.3010; Line Search: 1.8240
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -9.306957376572495 (> -Infinity) after 42.724s (< 5400.000s)
Returns
-9.306957376572495