BasicOptimizer.scala:89 executed in 37.75 seconds (0.694 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 636997182420800
Reset training subject: 636997799348900
Adding measurement 1332d702 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-15.347873471594099;dx=-1.0463053711262735E-6
New Minimum: -15.347873471594099 > -15.347876547233646
WOLFE (weak): th(2.154434690031884)=-15.347876547233646; dx=-1.045917288962796E-6 evalInputDelta=3.0756395474895726E-6
New Minimum: -15.347876547233646 > -15.347880853127977
WOLFE (weak): th(4.308869380063768)=-15.347880853127977; dx=-1.0467154302654114E-6 evalInputDelta=7.381533878358937E-6
New Minimum: -15.347880853127977 > -15.347896231312157
WOLFE (weak): th(12.926608140191302)=-15.347896231312157; dx=-1.0464789228140379E-6 evalInputDelta=2.27597180586514E-5
New Minimum: -15.347896231312157 > -15.347963895139497
WOLFE (weak): th(51.70643256076521)=-15.347963895139497; dx=-1.047779705761674E-6 evalInputDelta=9.042354539801067E-5
New Minimum: -15.347963895139497 > -15.34832558399386
WOLFE (weak): th(258.53216280382605)=-15.34832558399386; dx=-1.0461116065021165E-6 evalInputDelta=4.521123997616172E-4
New Minimum: -15.34832558399386 > -15.350587791313114
WOLFE (weak): th(1551.1929768229563)=-15.350587791313114; dx=-1.0488252562512759E-6 evalInputDelta=0.0027143197190149237
New Minimum: -15.350587791313114 > -15.36684521028734
WOLFE (weak): th(10858.350837760694)=-15.36684521028734; dx=-1.0455009984569472E-6 evalInputDelta=0.018971738693240425
New Minimum: -15.36684521028734 > -15.497736211558514
WOLFE (weak): th(86866.80670208555)=-15.497736211558514; dx=-1.0195062511413135E-6 evalInputDelta=0.14986273996441568
New Minimum: -15.497736211558514 > -16.53456560305442
END: th(781801.26031877)=-16.53456560305442; dx=-8.401779216487949E-7 evalInputDelta=1.1866921314603225
Fitness changed from -15.347873471594099 to -16.53456560305442
Iteration 1 complete. Error: -16.53456560305442 Total: 7.6296; Orientation: 0.0075; Line Search: 5.8832
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 2f99f64 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-16.53456560305442;dx=-9.175443934828133E-7
New Minimum: -16.53456560305442 > -18.11429192395831
END: th(1684339.7559414052)=-18.11429192395831; dx=-6.358288795512391E-7 evalInputDelta=1.5797263209038874
Fitness changed from -16.53456560305442 to -18.11429192395831
Iteration 2 complete. Error: -18.11429192395831 Total: 1.6756; Orientation: 0.0083; Line Search: 1.0677
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement cdf1065 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-18.11429192395831;dx=-8.269198411511622E-7
New Minimum: -18.11429192395831 > -20.50070575108052
END: th(3628800.0)=-20.50070575108052; dx=-4.735180861039836E-7 evalInputDelta=2.3864138271222117
Fitness changed from -18.11429192395831 to -20.50070575108052
Iteration 3 complete. Error: -20.50070575108052 Total: 1.6923; Orientation: 0.0071; Line Search: 1.1032
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 286aea69 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.339e+04, gradient 8.707e-04, dot -0.575; [481fff91-9080-469f-9d6f-49313ad9dae9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -20.50070575108052, -18.11429192395831, -16.53456560305442, -15.347873471594099
LBFGS Accumulation History: 3 points
Removed measurement 286aea69 to history. Total: 3
th(0)=-20.50070575108052;dx=-7.529724059389972E-7
New Minimum: -20.50070575108052 > -25.032325990992245
END: th(7818012.6031877)=-25.032325990992245; dx=-3.2937409305613104E-7 evalInputDelta=4.5316202399117245
Fitness changed from -20.50070575108052 to -25.032325990992245
Iteration 4 complete. Error: -25.032325990992245 Total: 1.8268; Orientation: 0.1459; Line Search: 1.1137
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 15c8fe43 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.612e+04, gradient 8.436e-04, dot -0.486; [481fff91-9080-469f-9d6f-49313ad9dae9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -25.032325990992245, -18.11429192395831, -16.53456560305442, -15.347873471594099
LBFGS Accumulation History: 3 points
Removed measurement 15c8fe43 to history. Total: 3
th(0)=-25.032325990992245;dx=-7.091898753862875E-7
New Minimum: -25.032325990992245 > -31.98922207587745
END: th(1.684339755941405E7)=-31.98922207587745; dx=-1.5242636396351256E-7 evalInputDelta=6.956896084885205
Fitness changed from -25.032325990992245 to -31.98922207587745
Iteration 5 complete. Error: -31.98922207587745 Total: 1.8246; Orientation: 0.1400; Line Search: 1.0850
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 7160f481 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.465e+04, gradient 7.229e-04, dot -0.350; [481fff91-9080-469f-9d6f-49313ad9dae9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -31.98922207587745, -18.11429192395831, -16.53456560305442, -15.347873471594099
LBFGS Accumulation History: 3 points
Removed measurement 7160f481 to history. Total: 3
th(0)=-31.98922207587745;dx=-5.200972048475616E-7
New Minimum: -31.98922207587745 > -41.129446923227924
END: th(3.6288E7)=-41.129446923227924; dx=-3.705721134195252E-8 evalInputDelta=9.140224847350474
Fitness changed from -31.98922207587745 to -41.129446923227924
Iteration 6 complete. Error: -41.129446923227924 Total: 1.9900; Orientation: 0.1352; Line Search: 1.1013
Adding measurement 77094d3b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.304e+04, gradient 5.791e-04, dot -0.225; [481fff91-9080-469f-9d6f-49313ad9dae9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -41.129446923227924, -18.11429192395831, -16.53456560305442, -15.347873471594099
LBFGS Accumulation History: 3 points
Removed measurement 77094d3b to history. Total: 3
th(0)=-41.129446923227924;dx=-3.329644347147371E-7
New Minimum: -41.129446923227924 > -51.62955289508194
WOLF (strong): th(7.8180126031877E7)=-51.62955289508194; dx=1.2142058151243178E-9 evalInputDelta=10.500105971854019
New Minimum: -51.62955289508194 > -56.89864621785219
END: th(3.90900630159385E7)=-56.89864621785219; dx=-4.4388897341782914E-8 evalInputDelta=15.769199294624265
Fitness changed from -41.129446923227924 to -56.89864621785219
Iteration 7 complete. Error: -56.89864621785219 Total: 2.3646; Orientation: 0.1416; Line Search: 1.6804
Adding measurement 469e48e6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.830e+04, gradient 4.444e-04, dot -0.204; [481fff91-9080-469f-9d6f-49313ad9dae9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -56.89864621785219, -18.11429192395831, -16.53456560305442, -15.347873471594099
LBFGS Accumulation History: 3 points
Removed measurement 469e48e6 to history. Total: 3
th(0)=-56.89864621785219;dx=-1.9625525144954502E-7
New Minimum: -56.89864621785219 > -62.231151739436264
END: th(8.421698779707026E7)=-62.231151739436264; dx=-6.330890420908863E-9 evalInputDelta=5.332505521584075
Fitness changed from -56.89864621785219 to -62.231151739436264
Iteration 8 complete. Error: -62.231151739436264 Total: 1.7964; Orientation: 0.1452; Line Search: 1.0898
Adding measurement 2720e99f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.353e+04, gradient 3.771e-04, dot -0.137; [481fff91-9080-469f-9d6f-49313ad9dae9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -62.231151739436264, -18.11429192395831, -16.53456560305442, -15.347873471594099
LBFGS Accumulation History: 3 points
Removed measurement 2720e99f to history. Total: 3
th(0)=-62.231151739436264;dx=-1.4116569398479275E-7
New Minimum: -62.231151739436264 > -65.78071207314196
WOLF (strong): th(1.8144000000000003E8)=-65.78071207314196; dx=3.4065159251273836E-9 evalInputDelta=3.549560333705692
New Minimum: -65.78071207314196 > -71.12343719367551
END: th(9.072000000000001E7)=-71.12343719367551; dx=-7.928999695490905E-9 evalInputDelta=8.89228545423925
Fitness changed from -62.231151739436264 to -71.12343719367551
Iteration 9 complete. Error: -71.12343719367551 Total: 2.3539; Orientation: 0.1382; Line Search: 1.6616
Adding measurement 428927d6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.160e+04, gradient 2.809e-04, dot -0.129; [481fff91-9080-469f-9d6f-49313ad9dae9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -71.12343719367551, -18.11429192395831, -16.53456560305442, -15.347873471594099
LBFGS Accumulation History: 3 points
Removed measurement 428927d6 to history. Total: 3
th(0)=-71.12343719367551;dx=-7.785646605332867E-8
New Minimum: -71.12343719367551 > -73.81892507351178
WOLF (strong): th(1.9545031507969254E8)=-73.81892507351178; dx=1.389595945211522E-9 evalInputDelta=2.6954878798362643
New Minimum: -73.81892507351178 > -77.06476737446816
END: th(9.772515753984627E7)=-77.06476737446816; dx=-8.681675106922528E-9 evalInputDelta=5.941330180792647
Fitness changed from -71.12343719367551 to -77.06476737446816
Iteration 10 complete. Error: -77.06476737446816 Total: 2.4797; Orientation: 0.1433; Line Search: 1.8054
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 2e235687 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.157e+04, gradient 2.277e-04, dot -0.137; [481fff91-9080-469f-9d6f-49313ad9dae9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -77.06476737446816, -18.11429192395831, -16.53456560305442, -15.347873471594099
LBFGS Accumulation History: 3 points
Removed measurement 2e235687 to history. Total: 3
th(0)=-77.06476737446816;dx=-5.1042290330722527E-8
Armijo: th(2.1054246949267572E8)=-75.87608351646287; dx=1.3591146298034003E-9 evalInputDelta=-1.188683858005291
New Minimum: -77.06476737446816 > -79.14914924434123
END: th(1.0527123474633786E8)=-79.14914924434123; dx=-5.978662661178738E-9 evalInputDelta=2.08438186987307
Fitness changed from -77.06476737446816 to -79.14914924434123
Iteration 11 complete. Error: -79.14914924434123 Total: 2.3948; Orientation: 0.1359; Line Search: 1.6715
Adding measurement 66599b2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.068e+04, gradient 1.908e-04, dot -0.138; [481fff91-9080-469f-9d6f-49313ad9dae9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -79.14914924434123, -18.11429192395831, -16.53456560305442, -15.347873471594099
LBFGS Accumulation History: 3 points
Removed measurement 66599b2 to history. Total: 3
th(0)=-79.14914924434123;dx=-3.567279405553784E-8
Armijo: th(2.268000000000001E8)=-78.5466469928918; dx=1.4759827769874608E-9 evalInputDelta=-0.6025022514494367
New Minimum: -79.14914924434123 > -81.03540029487425
END: th(1.1340000000000004E8)=-81.03540029487425; dx=-4.103081973118442E-9 evalInputDelta=1.8862510505330192
Fitness changed from -79.14914924434123 to -81.03540029487425
Iteration 12 complete. Error: -81.03540029487425 Total: 2.2973; Orientation: 0.1455; Line Search: 1.6135
Adding measurement 65dacb22 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.082e+04, gradient 1.688e-04, dot -0.144; [481fff91-9080-469f-9d6f-49313ad9dae9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -81.03540029487425, -18.11429192395831, -16.53456560305442, -15.347873471594099
LBFGS Accumulation History: 3 points
Removed measurement 65dacb22 to history. Total: 3
th(0)=-81.03540029487425;dx=-2.7847354967074878E-8
Armijo: th(2.4431289384961572E8)=-80.07952672121122; dx=1.3327780889895784E-9 evalInputDelta=-0.9558735736630268
New Minimum: -81.03540029487425 > -82.35763649786787
END: th(1.2215644692480786E8)=-82.35763649786787; dx=-2.771863425023343E-9 evalInputDelta=1.322236202993622
Fitness changed from -81.03540029487425 to -82.35763649786787
Iteration 13 complete. Error: -82.35763649786787 Total: 2.2991; Orientation: 0.1345; Line Search: 1.6082
Adding measurement 388e9ad6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.928e+03, gradient 1.491e-04, dot -0.141; [481fff91-9080-469f-9d6f-49313ad9dae9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -82.35763649786787, -18.11429192395831, -16.53456560305442, -15.347873471594099
LBFGS Accumulation History: 3 points
Removed measurement 388e9ad6 to history. Total: 3
th(0)=-82.35763649786787;dx=-2.1712517463358937E-8
Armijo: th(2.631780868658447E8)=-81.19619452119045; dx=1.4029513433355227E-9 evalInputDelta=-1.1614419766774233
New Minimum: -82.35763649786787 > -83.14369406737138
END: th(1.3158904343292235E8)=-83.14369406737138; dx=-2.033784297078827E-9 evalInputDelta=0.7860575695035124
Fitness changed from -82.35763649786787 to -83.14369406737138
Iteration 14 complete. Error: -83.14369406737138 Total: 2.5249; Orientation: 0.1389; Line Search: 1.8625
Adding measurement 5128509d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.410e+03, gradient 1.327e-04, dot -0.141; [481fff91-9080-469f-9d6f-49313ad9dae9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -83.14369406737138, -18.11429192395831, -16.53456560305442, -15.347873471594099
LBFGS Accumulation History: 3 points
Removed measurement 5128509d to history. Total: 3
th(0)=-83.14369406737138;dx=-1.7320935687536864E-8
Armijo: th(2.835000000000002E8)=-82.52651498825924; dx=7.838992323531382E-10 evalInputDelta=-0.6171790791121481
New Minimum: -83.14369406737138 > -84.15492067622658
END: th(1.417500000000001E8)=-84.15492067622658; dx=-1.8067492668473984E-9 evalInputDelta=1.0112266088551962
Fitness changed from -83.14369406737138 to -84.15492067622658
Iteration 15 complete. Error: -84.15492067622658 Total: 2.5443; Orientation: 0.1323; Line Search: 1.6122
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -84.15492067622658 (> -Infinity) after 37.749s (< 5400.000s)
Returns
-84.15492067622658