BasicOptimizer.scala:89 executed in 54.07 seconds (1.173 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 452037487644200
Reset training subject: 452038335357000
Adding measurement 2845e228 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-27.402589168516027;dx=-1.868543003214431E-5
New Minimum: -27.402589168516027 > -27.402636059815528
WOLFE (weak): th(2.154434690031884)=-27.402636059815528; dx=-1.8692536761265064E-5 evalInputDelta=4.689129950108395E-5
New Minimum: -27.402636059815528 > -27.402682951034787
WOLFE (weak): th(4.308869380063768)=-27.402682951034787; dx=-1.86690261264162E-5 evalInputDelta=9.378251876057675E-5
New Minimum: -27.402682951034787 > -27.402873049750312
WOLFE (weak): th(12.926608140191302)=-27.402873049750312; dx=-1.8619554599134303E-5 evalInputDelta=2.838812342851327E-4
New Minimum: -27.402873049750312 > -27.4037259430894
WOLFE (weak): th(51.70643256076521)=-27.4037259430894; dx=-1.8613576920705632E-5 evalInputDelta=0.0011367745733714685
New Minimum: -27.4037259430894 > -27.40827383696948
WOLFE (weak): th(258.53216280382605)=-27.40827383696948; dx=-1.860093272762927E-5 evalInputDelta=0.005684668453454833
New Minimum: -27.40827383696948 > -27.43669216953571
WOLFE (weak): th(1551.1929768229563)=-27.43669216953571; dx=-1.8668931354542816E-5 evalInputDelta=0.03410300101968389
New Minimum: -27.43669216953571 > -27.64084999470953
WOLFE (weak): th(10858.350837760694)=-27.64084999470953; dx=-1.8552264175744615E-5 evalInputDelta=0.2382608261935033
New Minimum: -27.64084999470953 > -29.28743920375144
WOLFE (weak): th(86866.80670208555)=-29.28743920375144; dx=-1.7892392188240912E-5 evalInputDelta=1.884850035235413
New Minimum: -29.28743920375144 > -38.53201052906232
END: th(781801.26031877)=-38.53201052906232; dx=-8.019529524307807E-6 evalInputDelta=11.129421360546292
Fitness changed from -27.402589168516027 to -38.53201052906232
Iteration 1 complete. Error: -38.53201052906232 Total: 10.7444; Orientation: 0.0122; Line Search: 8.0703
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 49ef8f10 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-38.53201052906232;dx=-1.7452418581825E-5
New Minimum: -38.53201052906232 > -45.933872094977254
END: th(1684339.7559414052)=-45.933872094977254; dx=-3.745592633799545E-6 evalInputDelta=7.401861565914935
Fitness changed from -38.53201052906232 to -45.933872094977254
Iteration 2 complete. Error: -45.933872094977254 Total: 2.2824; Orientation: 0.0123; Line Search: 1.4571
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 3c38c820 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-45.933872094977254;dx=-1.4900283906282623E-5
New Minimum: -45.933872094977254 > -50.62383662920882
END: th(3628800.0)=-50.62383662920882; dx=-1.6573139134445008E-6 evalInputDelta=4.689964534231564
Fitness changed from -45.933872094977254 to -50.62383662920882
Iteration 3 complete. Error: -50.62383662920882 Total: 2.2939; Orientation: 0.0126; Line Search: 1.5090
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 64eb19e0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.128e+04, gradient 3.280e-03, dot -0.254; [60b60c23-7ebc-454d-be5f-efed0233f1f7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -50.62383662920882, -45.933872094977254, -38.53201052906232, -27.402589168516027
LBFGS Accumulation History: 3 points
Removed measurement 64eb19e0 to history. Total: 3
th(0)=-50.62383662920882;dx=-1.0667869603350874E-5
New Minimum: -50.62383662920882 > -63.621869867102966
END: th(7818012.6031877)=-63.621869867102966; dx=-7.401585392810348E-7 evalInputDelta=12.998033237894148
Fitness changed from -50.62383662920882 to -63.621869867102966
Iteration 4 complete. Error: -63.621869867102966 Total: 3.0827; Orientation: 0.2996; Line Search: 1.6541
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 4716f2d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.805e+03, gradient 2.699e-03, dot -0.290; [60b60c23-7ebc-454d-be5f-efed0233f1f7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -63.621869867102966, -45.933872094977254, -38.53201052906232, -27.402589168516027
LBFGS Accumulation History: 3 points
Removed measurement 4716f2d to history. Total: 3
th(0)=-63.621869867102966;dx=-7.176219170018426E-6
New Minimum: -63.621869867102966 > -81.67152146337845
END: th(1.684339755941405E7)=-81.67152146337845; dx=-3.331209953382645E-7 evalInputDelta=18.04965159627549
Fitness changed from -63.621869867102966 to -81.67152146337845
Iteration 5 complete. Error: -81.67152146337845 Total: 2.6715; Orientation: 0.2823; Line Search: 1.4684
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 10f300e7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.533e+03, gradient 2.109e-03, dot -0.281; [60b60c23-7ebc-454d-be5f-efed0233f1f7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -81.67152146337845, -45.933872094977254, -38.53201052906232, -27.402589168516027
LBFGS Accumulation History: 3 points
Removed measurement 10f300e7 to history. Total: 3
th(0)=-81.67152146337845;dx=-4.401573089793246E-6
New Minimum: -81.67152146337845 > -107.30780563520327
END: th(3.6288E7)=-107.30780563520327; dx=-1.2336012470697823E-7 evalInputDelta=25.636284171824812
Fitness changed from -81.67152146337845 to -107.30780563520327
Iteration 6 complete. Error: -107.30780563520327 Total: 2.8376; Orientation: 0.2791; Line Search: 1.7470
Adding measurement 19313828 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.691e+03, gradient 1.600e-03, dot -0.157; [60b60c23-7ebc-454d-be5f-efed0233f1f7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -107.30780563520327, -45.933872094977254, -38.53201052906232, -27.402589168516027
LBFGS Accumulation History: 3 points
Removed measurement 19313828 to history. Total: 3
th(0)=-107.30780563520327;dx=-2.5365265442892354E-6
New Minimum: -107.30780563520327 > -128.82033187979482
END: th(7.8180126031877E7)=-128.82033187979482; dx=-1.6601368690328704E-8 evalInputDelta=21.512526244591555
Fitness changed from -107.30780563520327 to -128.82033187979482
Iteration 7 complete. Error: -128.82033187979482 Total: 2.6777; Orientation: 0.2518; Line Search: 1.6870
Adding measurement 46225650 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.041e+03, gradient 1.059e-03, dot -0.059; [60b60c23-7ebc-454d-be5f-efed0233f1f7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -128.82033187979482, -45.933872094977254, -38.53201052906232, -27.402589168516027
LBFGS Accumulation History: 3 points
Removed measurement 46225650 to history. Total: 3
th(0)=-128.82033187979482;dx=-1.1019613620230037E-6
New Minimum: -128.82033187979482 > -142.28844177736917
WOLF (strong): th(1.6843397559414053E8)=-142.28844177736917; dx=1.0879841876417834E-9 evalInputDelta=13.468109897574351
New Minimum: -142.28844177736917 > -150.72289030583946
END: th(8.421698779707026E7)=-150.72289030583946; dx=-3.527456834111843E-8 evalInputDelta=21.902558426044635
Fitness changed from -128.82033187979482 to -150.72289030583946
Iteration 8 complete. Error: -150.72289030583946 Total: 3.2899; Orientation: 0.2697; Line Search: 2.2440
Adding measurement 55fb67fb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.918e+04, gradient 7.191e-04, dot -0.036; [60b60c23-7ebc-454d-be5f-efed0233f1f7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -150.72289030583946, -45.933872094977254, -38.53201052906232, -27.402589168516027
LBFGS Accumulation History: 3 points
Removed measurement 55fb67fb to history. Total: 3
th(0)=-150.72289030583946;dx=-5.079569692453616E-7
New Minimum: -150.72289030583946 > -157.4798976898878
WOLF (strong): th(1.8144000000000003E8)=-157.4798976898878; dx=3.4224530542242253E-9 evalInputDelta=6.757007384048336
New Minimum: -157.4798976898878 > -163.44875538898387
END: th(9.072000000000001E7)=-163.44875538898387; dx=-2.3460354472517462E-8 evalInputDelta=12.725865083144413
Fitness changed from -150.72289030583946 to -163.44875538898387
Iteration 9 complete. Error: -163.44875538898387 Total: 3.4942; Orientation: 0.2254; Line Search: 2.2806
Adding measurement 1b8a3cd4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.487e+04, gradient 5.265e-04, dot -0.035; [60b60c23-7ebc-454d-be5f-efed0233f1f7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -163.44875538898387, -45.933872094977254, -38.53201052906232, -27.402589168516027
LBFGS Accumulation History: 3 points
Removed measurement 1b8a3cd4 to history. Total: 3
th(0)=-163.44875538898387;dx=-2.7136537843708545E-7
New Minimum: -163.44875538898387 > -164.5489328029283
WOLF (strong): th(1.9545031507969254E8)=-164.5489328029283; dx=3.833872456417594E-9 evalInputDelta=1.1001774139444365
New Minimum: -164.5489328029283 > -169.16059213886265
END: th(9.772515753984627E7)=-169.16059213886265; dx=-1.2034048202545567E-8 evalInputDelta=5.711836749878785
Fitness changed from -163.44875538898387 to -169.16059213886265
Iteration 10 complete. Error: -169.16059213886265 Total: 3.4004; Orientation: 0.4088; Line Search: 2.2882
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 53f33f06 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.203e+04, gradient 4.040e-04, dot -0.035; [60b60c23-7ebc-454d-be5f-efed0233f1f7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -169.16059213886265, -45.933872094977254, -38.53201052906232, -27.402589168516027
LBFGS Accumulation History: 3 points
Removed measurement 53f33f06 to history. Total: 3
th(0)=-169.16059213886265;dx=-1.6120334343435377E-7
New Minimum: -169.16059213886265 > -169.78250171715175
WOLF (strong): th(2.1054246949267572E8)=-169.78250171715175; dx=3.795501003745781E-9 evalInputDelta=0.6219095782890918
New Minimum: -169.78250171715175 > -173.29687914658916
END: th(1.0527123474633786E8)=-173.29687914658916; dx=-6.737388555132932E-9 evalInputDelta=4.136287007726509
Fitness changed from -169.16059213886265 to -173.29687914658916
Iteration 11 complete. Error: -173.29687914658916 Total: 3.5871; Orientation: 0.3685; Line Search: 2.3324
Adding measurement 1710b7b1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.956e+04, gradient 3.075e-04, dot -0.036; [60b60c23-7ebc-454d-be5f-efed0233f1f7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -173.29687914658916, -45.933872094977254, -38.53201052906232, -27.402589168516027
LBFGS Accumulation History: 3 points
Removed measurement 1710b7b1 to history. Total: 3
th(0)=-173.29687914658916;dx=-9.05530326930651E-8
Armijo: th(2.268000000000001E8)=-171.84202516045093; dx=2.689806788288182E-9 evalInputDelta=-1.4548539861382324
New Minimum: -173.29687914658916 > -174.71855585900127
END: th(1.1340000000000004E8)=-174.71855585900127; dx=-3.4965975724820343E-9 evalInputDelta=1.4216767124121077
Fitness changed from -173.29687914658916 to -174.71855585900127
Iteration 12 complete. Error: -174.71855585900127 Total: 3.3909; Orientation: 0.2460; Line Search: 2.4075
Adding measurement 703c4c76 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.528e+04, gradient 2.487e-04, dot -0.034; [60b60c23-7ebc-454d-be5f-efed0233f1f7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -174.71855585900127, -45.933872094977254, -38.53201052906232, -27.402589168516027
LBFGS Accumulation History: 3 points
Removed measurement 703c4c76 to history. Total: 3
th(0)=-174.71855585900127;dx=-6.045976746532978E-8
Armijo: th(2.4431289384961572E8)=-174.09209064832842; dx=1.6435852918849163E-9 evalInputDelta=-0.6264652106728477
New Minimum: -174.71855585900127 > -175.99826432027726
END: th(1.2215644692480786E8)=-175.99826432027726; dx=-1.5221723606009208E-9 evalInputDelta=1.2797084612759875
Fitness changed from -174.71855585900127 to -175.99826432027726
Iteration 13 complete. Error: -175.99826432027726 Total: 3.3997; Orientation: 0.2608; Line Search: 2.4103
Adding measurement 653f2b3c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.399e+04, gradient 2.013e-04, dot -0.034; [60b60c23-7ebc-454d-be5f-efed0233f1f7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -175.99826432027726, -45.933872094977254, -38.53201052906232, -27.402589168516027
LBFGS Accumulation History: 3 points
Removed measurement 653f2b3c to history. Total: 3
th(0)=-175.99826432027726;dx=-4.033695535152175E-8
Armijo: th(2.631780868658447E8)=-175.3541399757151; dx=1.2421704677135813E-9 evalInputDelta=-0.6441243445621581
New Minimum: -175.99826432027726 > -176.6379448280219
END: th(1.3158904343292235E8)=-176.6379448280219; dx=-9.637833573795194E-10 evalInputDelta=0.6396805077446288
Fitness changed from -175.99826432027726 to -176.6379448280219
Iteration 14 complete. Error: -176.6379448280219 Total: 3.1977; Orientation: 0.2322; Line Search: 2.2187
Adding measurement 31b767e0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.041e+04, gradient 1.625e-04, dot -0.033; [60b60c23-7ebc-454d-be5f-efed0233f1f7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -176.6379448280219, -45.933872094977254, -38.53201052906232, -27.402589168516027
LBFGS Accumulation History: 3 points
Removed measurement 31b767e0 to history. Total: 3
th(0)=-176.6379448280219;dx=-2.616348781601388E-8
Armijo: th(2.835000000000002E8)=-175.65946735013858; dx=9.542432754453268E-10 evalInputDelta=-0.9784774778833025
New Minimum: -176.6379448280219 > -176.8232871146007
END: th(1.417500000000001E8)=-176.8232871146007; dx=-5.352473794732338E-10 evalInputDelta=0.18534228657881613
Fitness changed from -176.6379448280219 to -176.8232871146007
Iteration 15 complete. Error: -176.8232871146007 Total: 3.6554; Orientation: 0.2503; Line Search: 2.6178
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -176.8232871146007 (> -Infinity) after 54.066s (< 5400.000s)
Returns
-176.8232871146007