BasicOptimizer.scala:89 executed in 58.35 seconds (1.240 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 209360809751300
Reset training subject: 209361590124400
Adding measurement 4a2e8b6e to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-7.1670724797546725;dx=-0.04000281102964553
New Minimum: -7.1670724797546725 > -7.26911797154677
WOLFE (weak): th(2.154434690031884)=-7.26911797154677; dx=-0.03949480228609882 evalInputDelta=0.10204549179209721
New Minimum: -7.26911797154677 > -7.3717804893164995
WOLFE (weak): th(4.308869380063768)=-7.3717804893164995; dx=-0.03908441156132854 evalInputDelta=0.204708009561827
New Minimum: -7.3717804893164995 > -7.788160867401501
WOLFE (weak): th(12.926608140191302)=-7.788160867401501; dx=-0.040840259311326566 evalInputDelta=0.6210883876468287
New Minimum: -7.788160867401501 > -9.755822173735414
WOLFE (weak): th(51.70643256076521)=-9.755822173735414; dx=-0.041770895219325886 evalInputDelta=2.5887496939807413
New Minimum: -9.755822173735414 > -22.00558297193971
WOLFE (weak): th(258.53216280382605)=-22.00558297193971; dx=-0.04789811584736757 evalInputDelta=14.838510492185037
New Minimum: -22.00558297193971 > -77.3270657687109
END: th(1551.1929768229563)=-77.3270657687109; dx=-0.02043663997080651 evalInputDelta=70.15999328895623
Fitness changed from -7.1670724797546725 to -77.3270657687109
Iteration 1 complete. Error: -77.3270657687109 Total: 7.7054; Orientation: 0.0247; Line Search: 5.4793
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 46dc1530 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-77.3270657687109;dx=-0.05473722588803434
New Minimum: -77.3270657687109 > -190.88135941716573
END: th(3341.943960201201)=-190.88135941716573; dx=-0.011638866766519013 evalInputDelta=113.55429364845483
Fitness changed from -77.3270657687109 to -190.88135941716573
Iteration 2 complete. Error: -190.88135941716573 Total: 2.1884; Orientation: 0.0155; Line Search: 1.4837
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 61a1cb70 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-190.88135941716573;dx=-0.03897127005835327
New Minimum: -190.88135941716573 > -349.17851159148313
END: th(7200.000000000001)=-349.17851159148313; dx=-0.007164146328449237 evalInputDelta=158.2971521743174
Fitness changed from -190.88135941716573 to -349.17851159148313
Iteration 3 complete. Error: -349.17851159148313 Total: 2.2903; Orientation: 0.0157; Line Search: 1.5071
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 6789984f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.423e+03, gradient 2.106e-01, dot -0.219; [f5d2413b-abde-4cda-8dbd-4d9bb0ffddfd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -349.17851159148313, -190.88135941716573, -77.3270657687109, -7.1670724797546725
LBFGS Accumulation History: 3 points
Removed measurement 6789984f to history. Total: 3
th(0)=-349.17851159148313;dx=-0.04426003101823851
New Minimum: -349.17851159148313 > -547.997357955907
END: th(15511.929768229566)=-547.997357955907; dx=-0.003120663045576716 evalInputDelta=198.81884636442385
Fitness changed from -349.17851159148313 to -547.997357955907
Iteration 4 complete. Error: -547.997357955907 Total: 2.5123; Orientation: 0.2913; Line Search: 1.5178
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement fe5a309 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.862e+03, gradient 2.071e-01, dot -0.160; [f5d2413b-abde-4cda-8dbd-4d9bb0ffddfd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -547.997357955907, -190.88135941716573, -77.3270657687109, -7.1670724797546725
LBFGS Accumulation History: 3 points
Removed measurement fe5a309 to history. Total: 3
th(0)=-547.997357955907;dx=-0.042770547433417495
New Minimum: -547.997357955907 > -671.1393389453392
END: th(33419.43960201202)=-671.1393389453392; dx=-8.402431691992482E-4 evalInputDelta=123.14198098943223
Fitness changed from -547.997357955907 to -671.1393389453392
Iteration 5 complete. Error: -671.1393389453392 Total: 2.7407; Orientation: 0.2859; Line Search: 1.4479
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 277d5e3e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.410e+03, gradient 2.005e-01, dot -0.124; [f5d2413b-abde-4cda-8dbd-4d9bb0ffddfd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -671.1393389453392, -190.88135941716573, -77.3270657687109, -7.1670724797546725
LBFGS Accumulation History: 3 points
Removed measurement 277d5e3e to history. Total: 3
th(0)=-671.1393389453392;dx=-0.04007370357887885
New Minimum: -671.1393389453392 > -750.403585175812
END: th(72000.00000000003)=-750.403585175812; dx=-1.5574842090224138E-4 evalInputDelta=79.26424623047285
Fitness changed from -671.1393389453392 to -750.403585175812
Iteration 6 complete. Error: -750.403585175812 Total: 2.5647; Orientation: 0.3019; Line Search: 1.4978
Adding measurement 3f1c88aa to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.522e+03, gradient 1.870e-01, dot -0.079; [f5d2413b-abde-4cda-8dbd-4d9bb0ffddfd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -750.403585175812, -190.88135941716573, -77.3270657687109, -7.1670724797546725
LBFGS Accumulation History: 3 points
Removed measurement 3f1c88aa to history. Total: 3
th(0)=-750.403585175812;dx=-0.034905226637417616
New Minimum: -750.403585175812 > -844.0910572800915
WOLF (strong): th(155119.2976822957)=-844.0910572800915; dx=1.3335384251692703E-4 evalInputDelta=93.68747210427944
New Minimum: -844.0910572800915 > -1141.3187732092458
END: th(77559.64884114785)=-1141.3187732092458; dx=-0.0016984514677985002 evalInputDelta=390.91518803343376
Fitness changed from -750.403585175812 to -1141.3187732092458
Iteration 7 complete. Error: -1141.3187732092458 Total: 3.6063; Orientation: 0.3005; Line Search: 2.5669
Adding measurement 77802e3f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.471e+04, gradient 1.661e-01, dot -0.103; [f5d2413b-abde-4cda-8dbd-4d9bb0ffddfd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1141.3187732092458, -190.88135941716573, -77.3270657687109, -7.1670724797546725
LBFGS Accumulation History: 3 points
Removed measurement 77802e3f to history. Total: 3
th(0)=-1141.3187732092458;dx=-0.02751012862347116
Armijo: th(167097.19801006012)=-1055.3407817694203; dx=3.290260660538625E-4 evalInputDelta=-85.97799143982547
New Minimum: -1141.3187732092458 > -1486.5229366965664
END: th(83548.59900503006)=-1486.5229366965664; dx=-0.0024507299501437645 evalInputDelta=345.20416348732056
Fitness changed from -1141.3187732092458 to -1486.5229366965664
Iteration 8 complete. Error: -1486.5229366965664 Total: 3.2137; Orientation: 0.2769; Line Search: 2.2111
Adding measurement 54a1551d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.638e+05, gradient 1.752e-01, dot -0.122; [f5d2413b-abde-4cda-8dbd-4d9bb0ffddfd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1486.5229366965664, -190.88135941716573, -77.3270657687109, -7.1670724797546725
LBFGS Accumulation History: 3 points
Removed measurement 54a1551d to history. Total: 3
th(0)=-1486.5229366965664;dx=-0.030637755414725257
Armijo: th(180000.0000000001)=-1201.1621416939602; dx=7.688697307142548E-4 evalInputDelta=-285.3607950026062
New Minimum: -1486.5229366965664 > -1770.5107293913888
END: th(90000.00000000004)=-1770.5107293913888; dx=-0.0018814950748288022 evalInputDelta=283.9877926948225
Fitness changed from -1486.5229366965664 to -1770.5107293913888
Iteration 9 complete. Error: -1770.5107293913888 Total: 3.4585; Orientation: 0.2873; Line Search: 2.4979
Adding measurement 7bfa2455 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 7.055e+04, gradient 1.792e-01, dot 0.138; [f5d2413b-abde-4cda-8dbd-4d9bb0ffddfd = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-1770.5107293913888;dx=-1749.1422194042505
Armijo: th(193899.12210286965)=-986.1542659243516; dx=0.004455977399137941 evalInputDelta=-784.3564634670372
Armijo: th(96949.56105143482)=-986.1270167129987; dx=0.00890685224623388 evalInputDelta=-784.3837126783901
Armijo: th(32316.520350478273)=-986.1669112682061; dx=0.026728439157059698 evalInputDelta=-784.3438181231827
Armijo: th(8079.130087619568)=-986.3132250158865; dx=0.1069597949048263 evalInputDelta=-784.1975043755024
Armijo: th(1615.8260175239136)=-986.7165844619451; dx=0.5331004907808798 evalInputDelta=-783.7941449294437
Armijo: th(269.3043362539856)=-992.597492395106; dx=3.1589877225723724 evalInputDelta=-777.9132369962829
Armijo: th(38.47204803628365)=-1029.5608343828912; dx=20.296106372275595 evalInputDelta=-740.9498950084976
Armijo: th(4.809006004535457)=-1166.884604971634; dx=108.13649475311658 evalInputDelta=-603.6261244197549
Armijo: th(0.5343340005039396)=-1610.5128588062141; dx=159.9269560289174 evalInputDelta=-159.99787058517472
New Minimum: -1770.5107293913888 > -1787.79222348513
END: th(0.053433400050393966)=-1787.79222348513; dx=-365.79733721988976 evalInputDelta=17.28149409374123
Fitness changed from -1770.5107293913888 to -1787.79222348513
Iteration 10 complete. Error: -1787.79222348513 Total: 9.4211; Orientation: 0.2715; Line Search: 8.4907
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 5b5f296 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 4.601e+05, gradient 1.760e-01, dot -0.097; [f5d2413b-abde-4cda-8dbd-4d9bb0ffddfd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1787.79222348513, -1770.5107293913888, -190.88135941716573, -77.3270657687109, -7.1670724797546725
Rejected: LBFGS Orientation magnitude: 9.492e+05, gradient 1.760e-01, dot -0.103; [f5d2413b-abde-4cda-8dbd-4d9bb0ffddfd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1787.79222348513, -1770.5107293913888, -190.88135941716573, -77.3270657687109
LBFGS Accumulation History: 3 points
Removed measurement 5b5f296 to history. Total: 4
Removed measurement 7bfa2455 to history. Total: 3
th(0)=-1787.79222348513;dx=-0.030934548154299118
New Minimum: -1787.79222348513 > -1787.794123211566
END: th(0.11511877067492017)=-1787.794123211566; dx=-0.023217942620029023 evalInputDelta=0.0018997264360223198
Fitness changed from -1787.79222348513 to -1787.794123211566
Iteration 11 complete. Error: -1787.794123211566 Total: 2.8923; Orientation: 0.6275; Line Search: 1.4769
Adding measurement 3e93ff06 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 9.236e+04, gradient 1.760e-01, dot 0.132; [f5d2413b-abde-4cda-8dbd-4d9bb0ffddfd = 1.000/1.000e+00]
th(0)=-1787.794123211566;dx=-2145.628767559011
Armijo: th(0.24801587301587316)=-1676.170522872516; dx=113.41860207544335 evalInputDelta=-111.6236003390502
Armijo: th(0.12400793650793658)=-1752.9340505470718; dx=-79.45560601002046 evalInputDelta=-34.86007266449428
Armijo: th(0.04133597883597886)=-1786.0565786524419; dx=-324.29053155527004 evalInputDelta=-1.737544559124217
New Minimum: -1787.794123211566 > -1788.7269917072545
END: th(0.010333994708994714)=-1788.7269917072545; dx=-447.41788724509195 evalInputDelta=0.9328684956883535
Fitness changed from -1787.794123211566 to -1788.7269917072545
Iteration 12 complete. Error: -1788.7269917072545 Total: 4.7938; Orientation: 0.2724; Line Search: 3.6765
Adding measurement 23819af5 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.076e+06, gradient 1.755e-01, dot -0.098; [f5d2413b-abde-4cda-8dbd-4d9bb0ffddfd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1788.7269917072545, -1787.794123211566, -190.88135941716573, -77.3270657687109, -7.1670724797546725
Rejected: LBFGS Orientation magnitude: 2.981e+06, gradient 1.755e-01, dot -0.102; [f5d2413b-abde-4cda-8dbd-4d9bb0ffddfd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1788.7269917072545, -1787.794123211566, -190.88135941716573, -77.3270657687109
LBFGS Accumulation History: 3 points
Removed measurement 23819af5 to history. Total: 4
Removed measurement 3e93ff06 to history. Total: 3
th(0)=-1788.7269917072545;dx=-0.030707810391224923
New Minimum: -1788.7269917072545 > -1788.727463483343
END: th(0.022263916687664156)=-1788.727463483343; dx=-0.02284981356692088 evalInputDelta=4.717760884886957E-4
Fitness changed from -1788.7269917072545 to -1788.727463483343
Iteration 13 complete. Error: -1788.727463483343 Total: 2.9760; Orientation: 0.6498; Line Search: 1.6627
Adding measurement 3db5cb4f to history. Total: 3
Accepted: LBFGS Orientation magnitude: 9.955e+04, gradient 1.755e-01, dot 0.130; [f5d2413b-abde-4cda-8dbd-4d9bb0ffddfd = 1.000/1.000e+00]
th(0)=-1788.727463483343;dx=-2269.4882399060552
Armijo: th(0.047966154447883415)=-1780.896568239667; dx=-284.20235619736957 evalInputDelta=-7.830895243675968
Armijo: th(0.023983077223941707)=-1786.9507360127575; dx=-383.0152679461256 evalInputDelta=-1.776727470585456
Armijo: th(0.007994359074647236)=-1788.6960810020635; dx=-457.80695391746565 evalInputDelta=-0.03138248127947918
New Minimum: -1788.727463483343 > -1788.7915282000595
END: th(0.001998589768661809)=-1788.7915282000595; dx=-484.6998696872762 evalInputDelta=0.0640647167165298
Fitness changed from -1788.727463483343 to -1788.7915282000595
Iteration 14 complete. Error: -1788.7915282000595 Total: 4.7778; Orientation: 0.2940; Line Search: 3.7614
Adding measurement 57e5546c to history. Total: 4
Rejected: LBFGS Orientation magnitude: 5.373e+07, gradient 1.751e-01, dot -0.101; [f5d2413b-abde-4cda-8dbd-4d9bb0ffddfd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1788.7915282000595, -1788.727463483343, -190.88135941716573, -77.3270657687109, -7.1670724797546725
Rejected: LBFGS Orientation magnitude: 1.958e+08, gradient 1.751e-01, dot -0.103; [f5d2413b-abde-4cda-8dbd-4d9bb0ffddfd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1788.7915282000595, -1788.727463483343, -190.88135941716573, -77.3270657687109
LBFGS Accumulation History: 3 points
Removed measurement 57e5546c to history. Total: 4
Removed measurement 3db5cb4f to history. Total: 3
th(0)=-1788.7915282000595;dx=-0.030529356622226955
New Minimum: -1788.7915282000595 > -1788.7915955783833
END: th(0.004305831128747798)=-1788.7915955783833; dx=-0.022739014250583292 evalInputDelta=6.737832381986664E-5
Fitness changed from -1788.7915282000595 to -1788.7915955783833
Iteration 15 complete. Error: -1788.7915955783833 Total: 3.1903; Orientation: 0.6776; Line Search: 1.7509
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -1788.7915955783833 (> -Infinity) after 58.354s (< 5400.000s)
Returns
-1788.7915955783833