BasicOptimizer.scala:88 executed in 273.88 seconds (7.873 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 1181309271698
Reset training subject: 1184415145409
Adding measurement 38c3362d to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=13.364936083555222;dx=-1.4515026032421357E-5
New Minimum: 13.364936083555222 > 13.364919662475586
WOLFE (weak): th(2.154434690031884)=13.364919662475586; dx=-1.4504129112180393E-5 evalInputDelta=1.6421079635620117E-5
New Minimum: 13.364919662475586 > 13.364903062582016
WOLFE (weak): th(4.308869380063768)=13.364903062582016; dx=-1.4503912874302388E-5 evalInputDelta=3.3020973205566406E-5
New Minimum: 13.364903062582016 > 13.36483895778656
WOLFE (weak): th(12.926608140191302)=13.36483895778656; dx=-1.450472977026008E-5 evalInputDelta=9.712576866149902E-5
New Minimum: 13.36483895778656 > 13.364548444747925
WOLFE (weak): th(51.70643256076521)=13.364548444747925; dx=-1.4503097949520416E-5 evalInputDelta=3.8763880729675293E-4
New Minimum: 13.364548444747925 > 13.362999230623245
WOLFE (weak): th(258.53216280382605)=13.362999230623245; dx=-1.4498475281998988E-5 evalInputDelta=0.0019368529319763184
New Minimum: 13.362999230623245 > 13.3533273935318
WOLFE (weak): th(1551.1929768229563)=13.3533273935318; dx=-1.4483491390206977E-5 evalInputDelta=0.011608690023422241
New Minimum: 13.3533273935318 > 13.284066766500473
WOLFE (weak): th(10858.350837760694)=13.284066766500473; dx=-1.4337455920671242E-5 evalInputDelta=0.08086931705474854
New Minimum: 13.284066766500473 > 12.77061128616333
END: th(86866.80670208555)=12.77061128616333; dx=-1.1677778622869811E-5 evalInputDelta=0.5943247973918915
Fitness changed from 13.364936083555222 to 12.77061128616333
Iteration 1 complete. Error: 12.77061128616333 Total: 37.1212; Orientation: 0.0937; Line Search: 27.9983
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 77d09595 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=12.77061128616333;dx=-1.101102658053036E-5
New Minimum: 12.77061128616333 > 11.87355974316597
END: th(187148.86177126726)=11.87355974316597; dx=-7.773182297647211E-6 evalInputDelta=0.8970515429973602
Fitness changed from 12.77061128616333 to 11.87355974316597
Iteration 2 complete. Error: 11.87355974316597 Total: 9.6291; Orientation: 0.0338; Line Search: 6.0525
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 5746447b to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=11.87355974316597;dx=-7.11804542721883E-6
New Minimum: 11.87355974316597 > 10.685363620519638
END: th(403200.00000000006)=10.685363620519638; dx=-4.611598076296231E-6 evalInputDelta=1.1881961226463318
Fitness changed from 11.87355974316597 to 10.685363620519638
Iteration 3 complete. Error: 10.685363620519638 Total: 8.8216; Orientation: 0.0294; Line Search: 5.9265
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 31778b9f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.235e+03, gradient 2.030e-03, dot -0.887; [7afb9154-f953-48a8-b2a2-af0864a3a580 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 10.685363620519638, 11.87355974316597, 12.77061128616333, 13.364936083555222
LBFGS Accumulation History: 3 points
Removed measurement 31778b9f to history. Total: 3
th(0)=10.685363620519638;dx=-4.121491714222549E-6
New Minimum: 10.685363620519638 > 9.28196282684803
END: th(868668.0670208557)=9.28196282684803; dx=-2.4018897961280964E-6 evalInputDelta=1.403400793671608
Fitness changed from 10.685363620519638 to 9.28196282684803
Iteration 4 complete. Error: 9.28196282684803 Total: 10.1209; Orientation: 0.7946; Line Search: 5.8447
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 1ed83b08 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.748e+03, gradient 1.490e-03, dot -0.812; [7afb9154-f953-48a8-b2a2-af0864a3a580 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 9.28196282684803, 11.87355974316597, 12.77061128616333, 13.364936083555222
LBFGS Accumulation History: 3 points
Removed measurement 1ed83b08 to history. Total: 3
th(0)=9.28196282684803;dx=-2.221267263445573E-6
New Minimum: 9.28196282684803 > 7.8533512353897095
END: th(1871488.617712673)=7.8533512353897095; dx=-9.596270265812095E-7 evalInputDelta=1.4286115914583206
Fitness changed from 9.28196282684803 to 7.8533512353897095
Iteration 5 complete. Error: 7.8533512353897095 Total: 9.8593; Orientation: 0.7854; Line Search: 6.0223
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 47e1eb00 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.172e+03, gradient 1.131e-03, dot -0.630; [7afb9154-f953-48a8-b2a2-af0864a3a580 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 7.8533512353897095, 11.87355974316597, 12.77061128616333, 13.364936083555222
LBFGS Accumulation History: 3 points
Removed measurement 47e1eb00 to history. Total: 3
th(0)=7.8533512353897095;dx=-1.280239299593807E-6
New Minimum: 7.8533512353897095 > 6.838228031992912
END: th(4032000.0000000014)=6.838228031992912; dx=-6.22333304973773E-8 evalInputDelta=1.0151232033967972
Fitness changed from 7.8533512353897095 to 6.838228031992912
Iteration 6 complete. Error: 6.838228031992912 Total: 10.2365; Orientation: 0.7640; Line Search: 5.8795
Adding measurement 83f602f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.169e+03, gradient 1.125e-03, dot -0.316; [7afb9154-f953-48a8-b2a2-af0864a3a580 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 6.838228031992912, 11.87355974316597, 12.77061128616333, 13.364936083555222
LBFGS Accumulation History: 3 points
Removed measurement 83f602f to history. Total: 3
th(0)=6.838228031992912;dx=-1.2662906588247965E-6
Armijo: th(8686680.670208558)=7.885909005999565; dx=7.675603679351716E-7 evalInputDelta=-1.0476809740066528
New Minimum: 6.838228031992912 > 6.5316465348005295
WOLF (strong): th(4343340.335104279)=6.5316465348005295; dx=4.7508845667549666E-7 evalInputDelta=0.3065814971923828
New Minimum: 6.5316465348005295 > 6.31751948595047
END: th(1447780.1117014263)=6.31751948595047; dx=-2.9735854516266566E-7 evalInputDelta=0.5207085460424423
Fitness changed from 6.838228031992912 to 6.31751948595047
Iteration 7 complete. Error: 6.31751948595047 Total: 16.6252; Orientation: 0.6927; Line Search: 12.9830
Adding measurement 30c22ca5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.640e+03, gradient 7.922e-04, dot -0.440; [7afb9154-f953-48a8-b2a2-af0864a3a580 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 6.31751948595047, 11.87355974316597, 12.77061128616333, 13.364936083555222
LBFGS Accumulation History: 3 points
Removed measurement 30c22ca5 to history. Total: 3
th(0)=6.31751948595047;dx=-6.276099310698544E-7
New Minimum: 6.31751948595047 > 6.031111001968384
WOLF (strong): th(3119147.6961877886)=6.031111001968384; dx=1.2554023058765932E-7 evalInputDelta=0.2864084839820862
New Minimum: 6.031111001968384 > 6.023259654641151
END: th(1559573.8480938943)=6.023259654641151; dx=-1.526672596948655E-7 evalInputDelta=0.29425983130931854
Fitness changed from 6.31751948595047 to 6.023259654641151
Iteration 8 complete. Error: 6.023259654641151 Total: 13.3959; Orientation: 0.7103; Line Search: 9.5873
Adding measurement 4f083a3e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.840e+03, gradient 7.953e-04, dot -0.434; [7afb9154-f953-48a8-b2a2-af0864a3a580 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 6.023259654641151, 11.87355974316597, 12.77061128616333, 13.364936083555222
LBFGS Accumulation History: 3 points
Removed measurement 4f083a3e to history. Total: 3
th(0)=6.023259654641151;dx=-6.324995838727887E-7
New Minimum: 6.023259654641151 > 5.876215174794197
WOLF (strong): th(3360000.0000000014)=5.876215174794197; dx=1.0156289721686782E-7 evalInputDelta=0.14704447984695435
New Minimum: 5.876215174794197 > 5.839124321937561
EN
...skipping 2464 bytes...
87355974316597, 12.77061128616333, 13.364936083555222
LBFGS Accumulation History: 3 points
Removed measurement 752b0442 to history. Total: 3
th(0)=5.542214050889015;dx=-3.1663314787296404E-7
New Minimum: 5.542214050889015 > 5.481244295835495
WOLF (strong): th(2800000.0000000014)=5.481244295835495; dx=8.721710973556522E-8 evalInputDelta=0.0609697550535202
New Minimum: 5.481244295835495 > 5.444563552737236
END: th(1400000.0000000007)=5.444563552737236; dx=-6.848813666688882E-9 evalInputDelta=0.09765049815177917
Fitness changed from 5.542214050889015 to 5.444563552737236
Iteration 12 complete. Error: 5.444563552737236 Total: 13.2846; Orientation: 0.6682; Line Search: 9.5435
Adding measurement 5e003ea4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.975e+03, gradient 7.444e-04, dot -0.396; [7afb9154-f953-48a8-b2a2-af0864a3a580 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 5.444563552737236, 11.87355974316597, 12.77061128616333, 13.364936083555222
LBFGS Accumulation History: 3 points
Removed measurement 5e003ea4 to history. Total: 3
th(0)=5.444563552737236;dx=-5.541238767098275E-7
Armijo: th(3016208.566044639)=5.501201286911964; dx=1.988805031423492E-7 evalInputDelta=-0.056637734174728394
New Minimum: 5.444563552737236 > 5.375137895345688
WOLF (strong): th(1508104.2830223194)=5.375137895345688; dx=1.2425218023443912E-7 evalInputDelta=0.06942565739154816
New Minimum: 5.375137895345688 > 5.356496587395668
END: th(502701.4276741065)=5.356496587395668; dx=-1.5292853282608392E-7 evalInputDelta=0.088066965341568
Fitness changed from 5.444563552737236 to 5.356496587395668
Iteration 13 complete. Error: 5.356496587395668 Total: 16.4951; Orientation: 0.6869; Line Search: 12.7560
Adding measurement 6248127d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.989e+03, gradient 4.435e-04, dot -0.621; [7afb9154-f953-48a8-b2a2-af0864a3a580 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 5.356496587395668, 11.87355974316597, 12.77061128616333, 13.364936083555222
LBFGS Accumulation History: 3 points
Removed measurement 6248127d to history. Total: 3
th(0)=5.356496587395668;dx=-1.9672125285905705E-7
New Minimum: 5.356496587395668 > 5.267455086112022
END: th(1083037.3945096491)=5.267455086112022; dx=-1.3628562782439753E-7 evalInputDelta=0.08904150128364563
Fitness changed from 5.356496587395668 to 5.267455086112022
Iteration 14 complete. Error: 5.267455086112022 Total: 9.8159; Orientation: 0.7143; Line Search: 6.1315
Adding measurement 40f60a7d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.992e+03, gradient 4.260e-04, dot -0.621; [7afb9154-f953-48a8-b2a2-af0864a3a580 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 5.267455086112022, 11.87355974316597, 12.77061128616333, 13.364936083555222
LBFGS Accumulation History: 3 points
Removed measurement 40f60a7d to history. Total: 3
th(0)=5.267455086112022;dx=-1.8148051883458205E-7
New Minimum: 5.267455086112022 > 5.131320551037788
END: th(2333333.333333335)=5.131320551037788; dx=-7.955392382750935E-8 evalInputDelta=0.136134535074234
Fitness changed from 5.267455086112022 to 5.131320551037788
Iteration 15 complete. Error: 5.131320551037788 Total: 10.3285; Orientation: 0.7189; Line Search: 6.6837
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Adding measurement 7fcbd6ac to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.897e+03, gradient 5.344e-04, dot -0.458; [7afb9154-f953-48a8-b2a2-af0864a3a580 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 5.131320551037788, 11.87355974316597, 12.77061128616333, 13.364936083555222
LBFGS Accumulation History: 3 points
Removed measurement 7fcbd6ac to history. Total: 3
th(0)=5.131320551037788;dx=-2.8555211589402444E-7
Armijo: th(5027014.276741066)=5.420794576406479; dx=2.5549862973043847E-7 evalInputDelta=-0.2894740253686905
New Minimum: 5.131320551037788 > 5.127629414200783
WOLF (strong): th(2513507.138370533)=5.127629414200783; dx=1.6963762398265932E-7 evalInputDelta=0.0036911368370056152
New Minimum: 5.127629414200783 > 5.058848097920418
END: th(837835.7127901777)=5.058848097920418; dx=-6.202851309119306E-8 evalInputDelta=0.0724724531173706
Fitness changed from 5.131320551037788 to 5.058848097920418
Iteration 16 complete. Error: 5.058848097920418 Total: 15.9250; Orientation: 0.6930; Line Search: 12.2750
Adding measurement 393e6584 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.945e+03, gradient 4.355e-04, dot -0.541; [7afb9154-f953-48a8-b2a2-af0864a3a580 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 5.058848097920418, 11.87355974316597, 12.77061128616333, 13.364936083555222
LBFGS Accumulation History: 3 points
Removed measurement 393e6584 to history. Total: 3
th(0)=5.058848097920418;dx=-1.8966499849830106E-7
New Minimum: 5.058848097920418 > 4.991960853338242
END: th(1805062.3241827488)=4.991960853338242; dx=-5.280418735484761E-9 evalInputDelta=0.06688724458217621
Fitness changed from 5.058848097920418 to 4.991960853338242
Iteration 17 complete. Error: 4.991960853338242 Total: 11.0990; Orientation: 0.6833; Line Search: 6.3172
Adding measurement 77a5c1ff to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.831e+03, gradient 6.388e-04, dot -0.363; [7afb9154-f953-48a8-b2a2-af0864a3a580 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 4.991960853338242, 11.87355974316597, 12.77061128616333, 13.364936083555222
LBFGS Accumulation History: 3 points
Removed measurement 77a5c1ff to history. Total: 3
th(0)=4.991960853338242;dx=-4.080219820297601E-7
Armijo: th(3888888.8888888923)=5.50728677213192; dx=4.3566801527991917E-7 evalInputDelta=-0.5153259187936783
Armijo: th(1944444.4444444461)=5.080247327685356; dx=3.7849037501014553E-7 evalInputDelta=-0.08828647434711456
New Minimum: 4.991960853338242 > 4.923415094614029
END: th(648148.1481481487)=4.923415094614029; dx=-1.2146142459646082E-8 evalInputDelta=0.06854575872421265
Fitness changed from 4.991960853338242 to 4.923415094614029
Iteration 18 complete. Error: 4.923415094614029 Total: 15.9807; Orientation: 0.6548; Line Search: 12.5142
Adding measurement 35fa8890 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.902e+03, gradient 4.207e-04, dot -0.515; [7afb9154-f953-48a8-b2a2-af0864a3a580 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 4.923415094614029, 11.87355974316597, 12.77061128616333, 13.364936083555222
LBFGS Accumulation History: 3 points
Removed measurement 35fa8890 to history. Total: 3
th(0)=4.923415094614029;dx=-1.7699795471160953E-7
New Minimum: 4.923415094614029 > 4.868637830018997
END: th(1396392.8546502963)=4.868637830018997; dx=-2.2800463457481262E-8 evalInputDelta=0.05477726459503174
Fitness changed from 4.923415094614029 to 4.868637830018997
Iteration 19 complete. Error: 4.868637830018997 Total: 9.7482; Orientation: 0.6754; Line Search: 6.0628
Adding measurement 23206369 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.857e+03, gradient 5.290e-04, dot -0.407; [7afb9154-f953-48a8-b2a2-af0864a3a580 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 4.868637830018997, 11.87355974316597, 12.77061128616333, 13.364936083555222
LBFGS Accumulation History: 3 points
Removed measurement 23206369 to history. Total: 3
th(0)=4.868637830018997;dx=-2.798358217047648E-7
Armijo: th(3008437.2069712486)=5.069809541106224; dx=2.938669773356165E-7 evalInputDelta=-0.20117171108722687
Armijo: th(1504218.6034856243)=4.87030665576458; dx=1.9435921418612576E-7 evalInputDelta=-0.0016688257455825806
New Minimum: 4.868637830018997 > 4.8252225667238235
END: th(501406.20116187475)=4.8252225667238235; dx=-6.546418606483543E-8 evalInputDelta=0.043415263295173645
Fitness changed from 4.868637830018997 to 4.8252225667238235
Iteration 20 complete. Error: 4.8252225667238235 Total: 15.9844; Orientation: 0.6910; Line Search: 11.9451
<a id="p-17"></a>Iteration 20
<a id="p-16"></a>
Final threshold in iteration 21: 4.8252225667238235 (> -Infinity) after 273.859s (< 5400.000s)
Returns
4.8252225667238235