BasicOptimizer.scala:89 executed in 50.94 seconds (0.996 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 216068712833500
Reset training subject: 216069352860300
Adding measurement 90a49f4 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-3.347879634274847;dx=-6.956349414696567E-5
New Minimum: -3.347879634274847 > -3.347985370037508
WOLFE (weak): th(2.154434690031884)=-3.347985370037508; dx=-6.956639568716068E-5 evalInputDelta=1.0573576266104112E-4
New Minimum: -3.347985370037508 > -3.3480915937767874
WOLFE (weak): th(4.308869380063768)=-3.3480915937767874; dx=-6.955788132583309E-5 evalInputDelta=2.1195950194030644E-4
New Minimum: -3.3480915937767874 > -3.3485154670436987
WOLFE (weak): th(12.926608140191302)=-3.3485154670436987; dx=-6.95265465552588E-5 evalInputDelta=6.358327688515963E-4
New Minimum: -3.3485154670436987 > -3.3504211597931906
WOLFE (weak): th(51.70643256076521)=-3.3504211597931906; dx=-6.954348728289429E-5 evalInputDelta=0.0025415255183434127
New Minimum: -3.3504211597931906 > -3.3605615066272514
WOLFE (weak): th(258.53216280382605)=-3.3605615066272514; dx=-6.952788573792509E-5 evalInputDelta=0.012681872352404255
New Minimum: -3.3605615066272514 > -3.423191837820003
WOLFE (weak): th(1551.1929768229563)=-3.423191837820003; dx=-6.926738122944322E-5 evalInputDelta=0.07531220354515566
New Minimum: -3.423191837820003 > -3.7986272462979676
END: th(10858.350837760694)=-3.7986272462979676; dx=-6.068173844023345E-5 evalInputDelta=0.4507476120231204
Fitness changed from -3.347879634274847 to -3.7986272462979676
Iteration 1 complete. Error: -3.7986272462979676 Total: 6.8147; Orientation: 0.0169; Line Search: 5.0022
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 2ee2b4ec to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-3.7986272462979676;dx=-8.431940559056962E-5
New Minimum: -3.7986272462979676 > -4.705633460831284
END: th(23393.607721408407)=-4.705633460831284; dx=-5.567278784960946E-5 evalInputDelta=0.9070062145333164
Fitness changed from -3.7986272462979676 to -4.705633460831284
Iteration 2 complete. Error: -4.705633460831284 Total: 1.8138; Orientation: 0.0161; Line Search: 1.1981
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 1d3953af to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-4.705633460831284;dx=-1.0117717525790522E-4
New Minimum: -4.705633460831284 > -5.941054908094196
END: th(50400.00000000001)=-5.941054908094196; dx=-4.0176579457479906E-5 evalInputDelta=1.2354214472629117
Fitness changed from -4.705633460831284 to -5.941054908094196
Iteration 3 complete. Error: -5.941054908094196 Total: 1.8316; Orientation: 0.0164; Line Search: 1.2351
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 3368568b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.632e+04, gradient 9.653e-03, dot -0.399; [886d6599-1350-4e39-8248-6947d164c460 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.941054908094196, -4.705633460831284, -3.7986272462979676, -3.347879634274847
LBFGS Accumulation History: 3 points
Removed measurement 3368568b to history. Total: 3
th(0)=-5.941054908094196;dx=-9.31487901271394E-5
New Minimum: -5.941054908094196 > -8.23506406513997
END: th(108583.50837760697)=-8.23506406513997; dx=-3.0746696756794635E-5 evalInputDelta=2.2940091570457737
Fitness changed from -5.941054908094196 to -8.23506406513997
Iteration 4 complete. Error: -8.23506406513997 Total: 2.1737; Orientation: 0.3072; Line Search: 1.2718
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 269cf1ab to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.427e+04, gradient 1.099e-02, dot -0.222; [886d6599-1350-4e39-8248-6947d164c460 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.23506406513997, -4.705633460831284, -3.7986272462979676, -3.347879634274847
LBFGS Accumulation History: 3 points
Removed measurement 269cf1ab to history. Total: 3
th(0)=-8.23506406513997;dx=-1.2067060919937987E-4
New Minimum: -8.23506406513997 > -8.70998210107928
END: th(233936.07721408413)=-8.70998210107928; dx=-7.660799989865174E-6 evalInputDelta=0.4749180359393108
Fitness changed from -8.23506406513997 to -8.70998210107928
Iteration 5 complete. Error: -8.70998210107928 Total: 2.0783; Orientation: 0.2995; Line Search: 1.2106
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 3a1c8f00 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.387e+03, gradient 1.217e-02, dot -0.095; [886d6599-1350-4e39-8248-6947d164c460 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.70998210107928, -4.705633460831284, -3.7986272462979676, -3.347879634274847
LBFGS Accumulation History: 3 points
Removed measurement 3a1c8f00 to history. Total: 3
th(0)=-8.70998210107928;dx=-1.4811189388779346E-4
Armijo: th(504000.0000000002)=-3.6126188590322776; dx=5.150214327017733E-6 evalInputDelta=-5.0973632420470025
New Minimum: -8.70998210107928 > -10.681761241346774
END: th(252000.0000000001)=-10.681761241346774; dx=-8.732834541643895E-6 evalInputDelta=1.9717791402674933
Fitness changed from -8.70998210107928 to -10.681761241346774
Iteration 6 complete. Error: -10.681761241346774 Total: 2.6814; Orientation: 0.3056; Line Search: 1.7790
Adding measurement 5695e434 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.451e+03, gradient 1.132e-02, dot -0.078; [886d6599-1350-4e39-8248-6947d164c460 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.681761241346774, -4.705633460831284, -3.7986272462979676, -3.347879634274847
LBFGS Accumulation History: 3 points
Removed measurement 5695e434 to history. Total: 3
th(0)=-10.681761241346774;dx=-1.2805568542210673E-4
Armijo: th(542917.5418880349)=-5.6129893450933235; dx=3.6693090137286988E-6 evalInputDelta=-5.06877189625345
New Minimum: -10.681761241346774 > -14.523818143318014
END: th(271458.77094401745)=-14.523818143318014; dx=-1.5353206060228017E-5 evalInputDelta=3.8420569019712403
Fitness changed from -10.681761241346774 to -14.523818143318014
Iteration 7 complete. Error: -14.523818143318014 Total: 2.8738; Orientation: 0.2817; Line Search: 1.7907
Adding measurement 6373088 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.776e+03, gradient 1.050e-02, dot -0.101; [886d6599-1350-4e39-8248-6947d164c460 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -14.523818143318014, -4.705633460831284, -3.7986272462979676, -3.347879634274847
LBFGS Accumulation History: 3 points
Removed measurement 6373088 to history. Total: 3
th(0)=-14.523818143318014;dx=-1.101706306425372E-4
Armijo: th(584840.1930352104)=-9.053854043255487; dx=2.4181432115399834E-6 evalInputDelta=-5.469964100062526
New Minimum: -14.523818143318014 > -18.69671143641983
END: th(292420.0965176052)=-18.69671143641983; dx=-1.7409671801297848E-5 evalInputDelta=4.172893293101815
Fitness changed from -14.523818143318014 to -18.69671143641983
Iteration 8 complete. Error: -18.69671143641983 Total: 2.8363; Orientation: 0.2946; Line Search: 2.0025
Adding measurement 574f7177 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.697e+04, gradient 1.057e-02, dot -0.111; [886d6599-1350-4e39-8248-6947d164c460 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.69671143641983, -4.705633460831284, -3.7986272462979676, -3.347879634274847
LBFGS Accumulation History: 3 points
Removed measurement 574f7177 to history. Total: 3
th(0)=-18.69671143641983;dx=-1.1164326306539678E-4
Armijo: th(630000.0000000002)=-10.471553744637532; dx=4.9333773383483845E-6 evalInputDelta=-8.225157691782297
New Minimum: -18.69671143641983 > -21.598858646164967
END: th(315000.0000000001)=-21.598858646164967; dx=-1.4861293859159166E-5 evalInputDelta=2.902147209745138
Fitness changed from -18.69671143641983 to -21.598858646164967
Iteration 9 complete. Error: -21.598858646164967 Total: 2.6689; Orientation: 0.2926; Line Search: 1.8283
Adding measurement 5fe6cb68 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.090e+04, gradient 1.060e-02, dot -0.121; [886d6599-1350-4e39-8248-6947d164c460 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -21.598858646164967, -4.705633460831284, -3.7986272462979676, -3.347879634274847
LBFGS Accumulation History: 3 points
Removed measurement 5fe6cb68 to history. Total: 3
th(0)=-21.598858646164967;dx=-1.1228220630007643E-4
Armijo: th(678646.9273600436)=-9.508999346579795; dx=8.695860378297498E-6 evalInputDelta=-12.089859299585171
New Minimum: -21.598858646164967 > -23.480264917377774
END: th(339323.4636800218)=-23.480264917377774; dx=-1.081853903665892E-5 evalInputDelta=1.881406271212807
Fitness changed from -21.598858646164967 to -23.480264917377774
Iteration 10 complete. Error: -23.480264917377774 Total: 2.9041; Orientation: 0.2925; Line Search: 2.0217
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 22087d76 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.994e+04, gradient 1.091e-02, dot -0.126; [886d6599-1350-4e39-8248-6947d164c460 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -23.480264917377774, -4.705633460831284, -3.7986272462979676, -3.347879634274847
LBFGS Accumulation History: 3 points
Removed measurement 22087d76 to history. Total: 3
th(0)=-23.480264917377774;dx=-1.1902791624074497E-4
Armijo: th(731050.241294013)=-8.460279496428534; dx=9.044144321798507E-6 evalInputDelta=-15.01998542094924
New Minimum: -23.480264917377774 > -24.557175119287756
END: th(365525.1206470065)=-24.557175119287756; dx=-5.579148924467527E-6 evalInputDelta=1.0769102019099819
Fitness changed from -23.480264917377774 to -24.557175119287756
Iteration 11 complete. Error: -24.557175119287756 Total: 2.7164; Orientation: 0.2900; Line Search: 1.8207
Adding measurement 2b540375 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.934e+04, gradient 1.122e-02, dot -0.108; [886d6599-1350-4e39-8248-6947d164c460 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -24.557175119287756, -4.705633460831284, -3.7986272462979676, -3.347879634274847
LBFGS Accumulation History: 3 points
Removed measurement 2b540375 to history. Total: 3
th(0)=-24.557175119287756;dx=-1.2590212793058252E-4
Armijo: th(787500.0000000003)=-7.793180601080046; dx=1.3731119317064539E-5 evalInputDelta=-16.76399451820771
New Minimum: -24.557175119287756 > -25.793505561157275
END: th(393750.0000000002)=-25.793505561157275; dx=-3.017248441617061E-6 evalInputDelta=1.2363304418695193
Fitness changed from -24.557175119287756 to -25.793505561157275
Iteration 12 complete. Error: -25.793505561157275 Total: 2.6923; Orientation: 0.3204; Line Search: 1.8383
Adding measurement 63a9729e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.373e+05, gradient 1.110e-02, dot -0.116; [886d6599-1350-4e39-8248-6947d164c460 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -25.793505561157275, -4.705633460831284, -3.7986272462979676, -3.347879634274847
LBFGS Accumulation History: 3 points
Removed measurement 63a9729e to history. Total: 3
th(0)=-25.793505561157275;dx=-1.2303152774862308E-4
Armijo: th(848308.6592000546)=-5.864864164383302; dx=1.3459630040221987E-5 evalInputDelta=-19.928641396773973
New Minimum: -25.793505561157275 > -26.391322410728186
END: th(424154.3296000273)=-26.391322410728186; dx=-3.619931810970926E-6 evalInputDelta=0.5978168495709113
Fitness changed from -25.793505561157275 to -26.391322410728186
Iteration 13 complete. Error: -26.391322410728186 Total: 2.6520; Orientation: 0.2903; Line Search: 1.8300
Adding measurement 8986541 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.104e+05, gradient 1.091e-02, dot -0.107; [886d6599-1350-4e39-8248-6947d164c460 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -26.391322410728186, -4.705633460831284, -3.7986272462979676, -3.347879634274847
LBFGS Accumulation History: 3 points
Removed measurement 8986541 to history. Total: 3
th(0)=-26.391322410728186;dx=-1.1905930253752556E-4
Armijo: th(913812.8016175163)=-6.037984688177872; dx=1.3753340815647184E-5 evalInputDelta=-20.353337722550314
New Minimum: -26.391322410728186 > -27.277101726764933
END: th(456906.40080875816)=-27.277101726764933; dx=-3.39940155915676E-6 evalInputDelta=0.8857793160367464
Fitness changed from -26.391322410728186 to -27.277101726764933
Iteration 14 complete. Error: -27.277101726764933 Total: 2.8926; Orientation: 0.2937; Line Search: 2.0527
Adding measurement 79c721e4 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 7.505e+04, gradient 1.094e-02, dot 0.122; [886d6599-1350-4e39-8248-6947d164c460 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-27.277101726764933;dx=-100.35368228287803
Armijo: th(984375.0000000006)=682.5228534234503; dx=2.9199848487182907E-4 evalInputDelta=-709.7999551502153
Armijo: th(492187.5000000003)=682.5196896395897; dx=5.840331081263989E-4 evalInputDelta=-709.7967913663547
Armijo: th(164062.5000000001)=682.5157904626476; dx=0.0017510696198548025 evalInputDelta=-709.7928921894126
Armijo: th(41015.62500000002)=682.449892843264; dx=0.007004735725365754 evalInputDelta=-709.726994570029
Armijo: th(8203.125000000004)=682.0326870480317; dx=0.03498921655711122 evalInputDelta=-709.3097887747966
Armijo: th(1367.1875000000007)=679.2783274600644; dx=0.20836989181906668 evalInputDelta=-706.5554291868293
Armijo: th(195.31250000000009)=663.3747945692294; dx=1.4090695413356753 evalInputDelta=-690.6518962959943
Armijo: th(24.41406250000001)=594.2042025150814; dx=9.79689273238543 evalInputDelta=-621.4813042418464
Armijo: th(2.7126736111111125)=343.28041127075426; dx=52.536660094341386 evalInputDelta=-370.5575129975192
Armijo: th(0.27126736111111127)=31.895374469433094; dx=71.21090236499367 evalInputDelta=-59.172476196198026
Armijo: th(0.024660669191919206)=-24.05457855380481; dx=-18.484412241838903 evalInputDelta=-3.2225231729601234
Armijo: th(0.002055055765993267)=-27.071158293868464; dx=-49.626472515584055 evalInputDelta=-0.20594343289646844
Armijo: th(1.5808121276871285E-4)=-27.26180262853306; dx=-52.70351699978792 evalInputDelta=-0.015299098231871255
Armijo: th(1.1291515197765203E-5)=-27.27601669091998; dx=-53.01744918122459 evalInputDelta=-0.0010850358449516762
Armijo: th(7.527676798510136E-7)=-27.277033376859833; dx=-53.001795078668714 evalInputDelta=-6.834990509929639E-5
New Minimum: -27.277101726764933 > -27.27710275959066
END: th(4.704797999068835E-8)=-27.27710275959066; dx=-52.911903811180146 evalInputDelta=1.0328257289415887E-6
Fitness changed from -27.277101726764933 to -27.27710275959066
Iteration 15 complete. Error: -27.27710275959066 Total: 11.2899; Orientation: 0.2789; Line Search: 10.4833
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -27.27710275959066 (> -Infinity) after 50.943s (< 5400.000s)
Returns
-27.27710275959066