BasicOptimizer.scala:89 executed in 54.95 seconds (1.289 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 213442958999100
Reset training subject: 213443558989100
Adding measurement 44ce89ea to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-35.96281377491721;dx=-0.020250362185014716
New Minimum: -35.96281377491721 > -35.98995806144676
WOLFE (weak): th(2.154434690031884)=-35.98995806144676; dx=-0.020045450408133676 evalInputDelta=0.027144286529548367
New Minimum: -35.98995806144676 > -36.01708815919204
WOLFE (weak): th(4.308869380063768)=-36.01708815919204; dx=-0.020153604860315293 evalInputDelta=0.054274384274833665
New Minimum: -36.01708815919204 > -36.12564804795826
WOLFE (weak): th(12.926608140191302)=-36.12564804795826; dx=-0.020081667134482818 evalInputDelta=0.16283427304104947
New Minimum: -36.12564804795826 > -36.61721374960155
WOLFE (weak): th(51.70643256076521)=-36.61721374960155; dx=-0.02045599766243392 evalInputDelta=0.6543999746843383
New Minimum: -36.61721374960155 > -39.3133887671395
WOLFE (weak): th(258.53216280382605)=-39.3133887671395; dx=-0.020959718662570688 evalInputDelta=3.350574992222292
New Minimum: -39.3133887671395 > -58.44896677111694
WOLFE (weak): th(1551.1929768229563)=-58.44896677111694; dx=-0.02090922181630326 evalInputDelta=22.486152996199735
New Minimum: -58.44896677111694 > -143.84607023955155
END: th(10858.350837760694)=-143.84607023955155; dx=-0.004243072809184601 evalInputDelta=107.88325646463434
Fitness changed from -35.96281377491721 to -143.84607023955155
Iteration 1 complete. Error: -143.84607023955155 Total: 6.9852; Orientation: 0.0169; Line Search: 5.2259
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 34550e28 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-143.84607023955155;dx=-0.036565237857183436
New Minimum: -143.84607023955155 > -289.9236837993347
END: th(23393.607721408407)=-289.9236837993347; dx=-0.0012259256287501844 evalInputDelta=146.07761355978315
Fitness changed from -143.84607023955155 to -289.9236837993347
Iteration 2 complete. Error: -289.9236837993347 Total: 1.8492; Orientation: 0.0168; Line Search: 1.2412
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 5e758f6e to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-289.9236837993347;dx=-0.030367601914028416
New Minimum: -289.9236837993347 > -299.7414231927456
WOLF (strong): th(50400.00000000001)=-299.7414231927456; dx=9.112165059175054E-4 evalInputDelta=9.817739393410875
New Minimum: -299.7414231927456 > -444.59522387280225
END: th(25200.000000000004)=-444.59522387280225; dx=-0.0029119735787197587 evalInputDelta=154.67154007346755
Fitness changed from -289.9236837993347 to -444.59522387280225
Iteration 3 complete. Error: -444.59522387280225 Total: 2.4617; Orientation: 0.0168; Line Search: 1.8439
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 4d840693 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.518e+03, gradient 1.755e-01, dot -0.271; [8479e254-8d3f-4f40-b245-d42a7772dbf1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -444.59522387280225, -289.9236837993347, -143.84607023955155, -35.96281377491721
LBFGS Accumulation History: 3 points
Removed measurement 4d840693 to history. Total: 3
th(0)=-444.59522387280225;dx=-0.030697312041577343
Armijo: th(54291.75418880348)=-337.6298049822344; dx=6.075003566064943E-4 evalInputDelta=-106.96541889056783
New Minimum: -444.59522387280225 > -571.6504876369462
END: th(27145.87709440174)=-571.6504876369462; dx=-0.004306637980369736 evalInputDelta=127.05526376414394
Fitness changed from -444.59522387280225 to -571.6504876369462
Iteration 4 complete. Error: -571.6504876369462 Total: 2.7028; Orientation: 0.2954; Line Search: 1.8210
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 3e0a65ea to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.272e+03, gradient 1.814e-01, dot -0.252; [8479e254-8d3f-4f40-b245-d42a7772dbf1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -571.6504876369462, -289.9236837993347, -143.84607023955155, -35.96281377491721
LBFGS Accumulation History: 3 points
Removed measurement 3e0a65ea to history. Total: 3
th(0)=-571.6504876369462;dx=-0.032711015096880944
Armijo: th(58484.01930352103)=-404.9007977672113; dx=8.537463228927606E-4 evalInputDelta=-166.74968986973488
New Minimum: -571.6504876369462 > -707.7793973221186
END: th(29242.009651760516)=-707.7793973221186; dx=-0.005245965391005427 evalInputDelta=136.1289096851724
Fitness changed from -571.6504876369462 to -707.7793973221186
Iteration 5 complete. Error: -707.7793973221186 Total: 2.9223; Orientation: 0.2917; Line Search: 2.0678
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 6724b34b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.414e+03, gradient 1.851e-01, dot -0.208; [8479e254-8d3f-4f40-b245-d42a7772dbf1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -707.7793973221186, -289.9236837993347, -143.84607023955155, -35.96281377491721
LBFGS Accumulation History: 3 points
Removed measurement 6724b34b to history. Total: 3
th(0)=-707.7793973221186;dx=-0.03418365525685156
Armijo: th(63000.00000000002)=-491.4062965667807; dx=0.0012706328883077323 evalInputDelta=-216.37310075533787
New Minimum: -707.7793973221186 > -846.9927802249975
END: th(31500.00000000001)=-846.9927802249975; dx=-0.005557136611351906 evalInputDelta=139.21338290287895
Fitness changed from -707.7793973221186 to -846.9927802249975
Iteration 6 complete. Error: -846.9927802249975 Total: 2.7827; Orientation: 0.2943; Line Search: 1.9135
Adding measurement 3ea90817 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.477e+04, gradient 1.891e-01, dot -0.186; [8479e254-8d3f-4f40-b245-d42a7772dbf1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -846.9927802249975, -289.9236837993347, -143.84607023955155, -35.96281377491721
LBFGS Accumulation History: 3 points
Removed measurement 3ea90817 to history. Total: 3
th(0)=-846.9927802249975;dx=-0.035646589374198887
Armijo: th(67864.69273600436)=-524.4867123007004; dx=0.0011896511846984628 evalInputDelta=-322.5060679242971
New Minimum: -846.9927802249975 > -967.3444775882494
END: th(33932.34636800218)=-967.3444775882494; dx=-0.005797922053426791 evalInputDelta=120.3516973632519
Fitness changed from -846.9927802249975 to -967.3444775882494
Iteration 7 complete. Error: -967.3444775882494 Total: 2.7226; Orientation: 0.2956; Line Search: 1.8348
Adding measurement 1cebf659 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.259e+04, gradient 1.996e-01, dot -0.166; [8479e254-8d3f-4f40-b245-d42a7772dbf1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -967.3444775882494, -289.9236837993347, -143.84607023955155, -35.96281377491721
LBFGS Accumulation History: 3 points
Removed measurement 1cebf659 to history. Total: 3
th(0)=-967.3444775882494;dx=-0.03972179434913412
Armijo: th(73105.0241294013)=-517.4272429568052; dx=0.001432941015156155 evalInputDelta=-449.9172346314442
New Minimum: -967.3444775882494 > -1041.7955640339176
END: th(36552.51206470065)=-1041.7955640339176; dx=-0.0050664662842778395 evalInputDelta=74.45108644566812
Fitness changed from -967.3444775882494 to -1041.7955640339176
Iteration 8 complete. Error: -1041.7955640339176 Total: 2.9947; Orientation: 0.3122; Line Search: 2.0777
Adding measurement 9d51ac2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.084e+04, gradient 2.069e-01, dot -0.154; [8479e254-8d3f-4f40-b245-d42a7772dbf1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1041.7955640339176, -289.9236837993347, -143.84607023955155, -35.96281377491721
LBFGS Accumulation History: 3 points
Removed measurement 9d51ac2 to history. Total: 3
th(0)=-1041.7955640339176;dx=-0.04258153395138652
Armijo: th(78750.00000000003)=-473.70495147433417; dx=0.0022394436155176193 evalInputDelta=-568.0906125595834
New Minimum: -1041.7955640339176 > -1109.9720471474166
END: th(39375.000000000015)=-1109.9720471474166; dx=-0.004889569570468367 evalInputDelta=68.17648311349899
Fitness changed from -1041.7955640339176 to -1109.9720471474166
Iteration 9 complete. Error: -1109.9720471474166 Total: 3.0722; Orientation: 0.3343; Line Search: 2.1585
Adding measurement 5dc14d6d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.661e+05, gradient 2.151e-01, dot -0.137; [8479e254-8d3f-4f40-b245-d42a7772dbf1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1109.9720471474166, -289.9236837993347, -143.84607023955155, -35.96281377491721
LBFGS Accumulation History: 3 points
Removed measurement 5dc14d6d to history. Total: 3
th(0)=-1109.9720471474166;dx=-0.04614319966561453
Armijo: th(84830.86592000545)=-360.95199356044293; dx=0.00265504063559545 evalInputDelta=-749.0200535869736
New Minimum: -1109.9720471474166 > -1112.9081651253136
END: th(42415.43296000273)=-1112.9081651253136; dx=-0.004302662841375014 evalInputDelta=2.9361179778970836
Fitness changed from -1109.9720471474166 to -1112.9081651253136
Iteration 10 complete. Error: -1112.9081651253136 Total: 2.7173; Orientation: 0.2828; Line Search: 1.8505
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 731cd4ad to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.084e+08, gradient 2.242e-01, dot -0.121; [8479e254-8d3f-4f40-b245-d42a7772dbf1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1112.9081651253136, -289.9236837993347, -143.84607023955155, -35.96281377491721
LBFGS Accumulation History: 3 points
Removed measurement 731cd4ad to history. Total: 3
th(0)=-1112.9081651253136;dx=-0.05011437177479934
Armijo: th(91381.28016175162)=-385.08895451946614; dx=0.0031308242552214897 evalInputDelta=-727.8192106058475
New Minimum: -1112.9081651253136 > -1158.7920371000748
END: th(45690.64008087581)=-1158.7920371000748; dx=-0.003103395192944638 evalInputDelta=45.88387197476118
Fitness changed from -1112.9081651253136 to -1158.7920371000748
Iteration 11 complete. Error: -1158.7920371000748 Total: 2.7659; Orientation: 0.3097; Line Search: 1.8881
Adding measurement 11194eb9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.792e+04, gradient 2.269e-01, dot -0.076; [8479e254-8d3f-4f40-b245-d42a7772dbf1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1158.7920371000748, -289.9236837993347, -143.84607023955155, -35.96281377491721
LBFGS Accumulation History: 3 points
Removed measurement 11194eb9 to history. Total: 3
th(0)=-1158.7920371000748;dx=-0.0513192547789297
Armijo: th(98437.50000000004)=-328.8525261245958; dx=0.0032910251876836854 evalInputDelta=-829.939510975479
New Minimum: -1158.7920371000748 > -1217.2438634239038
END: th(49218.75000000002)=-1217.2438634239038; dx=-0.0038159529698209385 evalInputDelta=58.45182632382898
Fitness changed from -1158.7920371000748 to -1217.2438634239038
Iteration 12 complete. Error: -1217.2438634239038 Total: 2.7153; Orientation: 0.2879; Line Search: 1.8643
Adding measurement 3a3245b5 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 3.304e+03, gradient 2.257e-01, dot 0.407; [8479e254-8d3f-4f40-b245-d42a7772dbf1 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-1217.2438634239038;dx=-303.4157767761551
Armijo: th(106038.58240000682)=3791.552267224392; dx=0.0225165523494624 evalInputDelta=-5008.796130648296
Armijo: th(53019.29120000341)=3706.897011164498; dx=0.043890152691688986 evalInputDelta=-4924.140874588402
Armijo: th(17673.097066667804)=3550.564467499639; dx=0.12456114174210761 evalInputDelta=-4767.808330923543
Armijo: th(4418.274266666951)=3283.897926939081; dx=0.45924151759744103 evalInputDelta=-4501.1417903629845
Armijo: th(883.6548533333902)=2840.377143865565; dx=1.918546347501556 evalInputDelta=-4057.6210072894687
Armijo: th(147.27580888889835)=1943.7145870417326; dx=7.4011010633043455 evalInputDelta=-3160.9584504656364
Armijo: th(21.03940126984262)=499.3848466860089; dx=16.32118871232511 evalInputDelta=-1716.6287101099128
Armijo: th(2.6299251587303276)=-1064.2719321127033; dx=-37.29590168551991 evalInputDelta=-152.97193131120048
New Minimum: -1217.2438634239038 > -1252.5724543134647
END: th(0.292213906525592)=-1252.5724543134647; dx=-196.97918892661835 evalInputDelta=35.32859088956093
Fitness changed from -1217.2438634239038 to -1252.5724543134647
Iteration 13 complete. Error: -1252.5724543134647 Total: 7.5603; Orientation: 0.2899; Line Search: 6.7168
Adding measurement 534edc26 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.032e+06, gradient 2.225e-01, dot -0.098; [8479e254-8d3f-4f40-b245-d42a7772dbf1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1252.5724543134647, -1217.2438634239038, -289.9236837993347, -143.84607023955155, -35.96281377491721
Rejected: LBFGS Orientation magnitude: 2.373e+06, gradient 2.225e-01, dot -0.098; [8479e254-8d3f-4f40-b245-d42a7772dbf1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1252.5724543134647, -1217.2438634239038, -289.9236837993347, -143.84607023955155
LBFGS Accumulation History: 3 points
Removed measurement 534edc26 to history. Total: 4
Removed measurement 3a3245b5 to history. Total: 3
th(0)=-1252.5724543134647;dx=-0.049200149839482885
New Minimum: -1252.5724543134647 > -1252.588821937412
END: th(0.6295557771284697)=-1252.588821937412; dx=-0.03870830382573271 evalInputDelta=0.016367623947189713
Fitness changed from -1252.5724543134647 to -1252.588821937412
Iteration 14 complete. Error: -1252.588821937412 Total: 2.4032; Orientation: 0.6444; Line Search: 1.1954
Adding measurement 5e263236 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.530e+04, gradient 2.222e-01, dot 0.186; [8479e254-8d3f-4f40-b245-d42a7772dbf1 = 1.000/1.000e+00]
th(0)=-1252.588821937412;dx=-631.7336252856103
Armijo: th(1.3563368055555562)=-582.29513099633; dx=131.6351489161013 evalInputDelta=-670.2936909410819
Armijo: th(0.6781684027777781)=-938.1865049863729; dx=64.68743460670058 evalInputDelta=-314.402316951039
Armijo: th(0.22605613425925938)=-1174.5730790355813; dx=-51.577656209202125 evalInputDelta=-78.01574290183066
Armijo: th(0.056514033564814846)=-1240.4319583563772; dx=-122.91563343227605 evalInputDelta=-12.156863581034713
Armijo: th(0.011302806712962968)=-1251.09188279857; dx=-170.55801125940872 evalInputDelta=-1.4969391388419808
Armijo: th(0.0018838011188271615)=-1252.3892014665155; dx=-192.44634543670085 evalInputDelta=-0.1996204708964342
Armijo: th(2.6911444554673735E-4)=-1252.5623669123436; dx=-197.24618341950236 evalInputDelta=-0.026455025068344185
Armijo: th(3.363930569334217E-5)=-1252.585563813984; dx=-190.95967602413072 evalInputDelta=-0.003258123427940518
Armijo: th(3.7377006325935744E-6)=-1252.5885125759385; dx=-171.9598077351099 evalInputDelta=-3.09361473455283E-4
New Minimum: -1252.588821937412 > -1252.588835644617
END: th(3.737700632593574E-7)=-1252.588835644617; dx=-174.07997322434755 evalInputDelta=1.3707204971069586E-5
Fitness changed from -1252.588821937412 to -1252.588835644617
Iteration 15 complete. Error: -1252.588835644617 Total: 8.2676; Orientation: 0.3135; Line Search: 7.3439
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -1252.588835644617 (> -Infinity) after 54.946s (< 5400.000s)
Returns
-1252.588835644617