BasicOptimizer.scala:89 executed in 39.12 seconds (0.974 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 566420292136300
Reset training subject: 566420962986100
Adding measurement 4310c90 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-20.64304255626196;dx=-1.3500164627227817E-6
New Minimum: -20.64304255626196 > -20.643045300306774
WOLFE (weak): th(2.154434690031884)=-20.643045300306774; dx=-1.3545832506686647E-6 evalInputDelta=2.744044813596247E-6
New Minimum: -20.643045300306774 > -20.643048044351225
WOLFE (weak): th(4.308869380063768)=-20.643048044351225; dx=-1.348279654911312E-6 evalInputDelta=5.488089264815699E-6
New Minimum: -20.643048044351225 > -20.64305993520629
WOLFE (weak): th(12.926608140191302)=-20.64305993520629; dx=-1.3513521730393124E-6 evalInputDelta=1.737894432807252E-5
New Minimum: -20.64305993520629 > -20.643112071951478
WOLFE (weak): th(51.70643256076521)=-20.643112071951478; dx=-1.3526281341025314E-6 evalInputDelta=6.951568951762965E-5
New Minimum: -20.643112071951478 > -20.643391047034875
WOLFE (weak): th(258.53216280382605)=-20.643391047034875; dx=-1.354264970869398E-6 evalInputDelta=3.484907729145448E-4
New Minimum: -20.643391047034875 > -20.645138900194798
WOLFE (weak): th(1551.1929768229563)=-20.645138900194798; dx=-1.352428159579712E-6 evalInputDelta=0.002096343932837641
New Minimum: -20.645138900194798 > -20.65772255066799
WOLFE (weak): th(10858.350837760694)=-20.65772255066799; dx=-1.3507990781322327E-6 evalInputDelta=0.014679994406030517
New Minimum: -20.65772255066799 > -20.76008180588128
WOLFE (weak): th(86866.80670208555)=-20.76008180588128; dx=-1.3446294112169652E-6 evalInputDelta=0.11703924961932088
New Minimum: -20.76008180588128 > -21.626913324799613
END: th(781801.26031877)=-21.626913324799613; dx=-1.1998826147539082E-6 evalInputDelta=0.9838707685376527
Fitness changed from -20.64304255626196 to -21.626913324799613
Iteration 1 complete. Error: -21.626913324799613 Total: 7.9772; Orientation: 0.0071; Line Search: 5.6293
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 6687afa0 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-21.626913324799613;dx=-1.3189241015280127E-6
New Minimum: -21.626913324799613 > -23.491595776006626
END: th(1684339.7559414052)=-23.491595776006626; dx=-1.0198152557376244E-6 evalInputDelta=1.8646824512070133
Fitness changed from -21.626913324799613 to -23.491595776006626
Iteration 2 complete. Error: -23.491595776006626 Total: 1.7464; Orientation: 0.0077; Line Search: 1.1148
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 35cf1aa0 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-23.491595776006626;dx=-1.215807111013013E-6
New Minimum: -23.491595776006626 > -26.231101689361083
END: th(3628800.0)=-26.231101689361083; dx=-7.751897779293068E-7 evalInputDelta=2.7395059133544564
Fitness changed from -23.491595776006626 to -26.231101689361083
Iteration 3 complete. Error: -26.231101689361083 Total: 1.9833; Orientation: 0.0072; Line Search: 1.3720
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 7983f244 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.201e+04, gradient 1.029e-03, dot -0.588; [64f6be10-f521-4077-9714-14c3daf0cbdb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -26.231101689361083, -23.491595776006626, -21.626913324799613, -20.64304255626196
LBFGS Accumulation History: 3 points
Removed measurement 7983f244 to history. Total: 3
th(0)=-26.231101689361083;dx=-1.0541043264815088E-6
New Minimum: -26.231101689361083 > -30.95488910703075
END: th(7818012.6031877)=-30.95488910703075; dx=-5.080043814087786E-7 evalInputDelta=4.723787417669666
Fitness changed from -26.231101689361083 to -30.95488910703075
Iteration 4 complete. Error: -30.95488910703075 Total: 1.9307; Orientation: 0.1440; Line Search: 1.1587
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 1b57cbb9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.928e+04, gradient 9.058e-04, dot -0.452; [64f6be10-f521-4077-9714-14c3daf0cbdb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -30.95488910703075, -23.491595776006626, -21.626913324799613, -20.64304255626196
LBFGS Accumulation History: 3 points
Removed measurement 1b57cbb9 to history. Total: 3
th(0)=-30.95488910703075;dx=-8.118004965622267E-7
New Minimum: -30.95488910703075 > -39.3080339061763
END: th(1.684339755941405E7)=-39.3080339061763; dx=-2.5124771522978204E-7 evalInputDelta=8.35314479914555
Fitness changed from -30.95488910703075 to -39.3080339061763
Iteration 5 complete. Error: -39.3080339061763 Total: 1.8937; Orientation: 0.1442; Line Search: 1.1301
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 74380738 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.460e+04, gradient 7.423e-04, dot -0.320; [64f6be10-f521-4077-9714-14c3daf0cbdb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -39.3080339061763, -23.491595776006626, -21.626913324799613, -20.64304255626196
LBFGS Accumulation History: 3 points
Removed measurement 74380738 to history. Total: 3
th(0)=-39.3080339061763;dx=-5.473536005997454E-7
New Minimum: -39.3080339061763 > -48.63084831048002
END: th(3.6288E7)=-48.63084831048002; dx=-7.418250206319587E-8 evalInputDelta=9.32281440430372
Fitness changed from -39.3080339061763 to -48.63084831048002
Iteration 6 complete. Error: -48.63084831048002 Total: 1.8781; Orientation: 0.1483; Line Search: 1.1246
Adding measurement 7f244c9f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.666e+04, gradient 5.790e-04, dot -0.205; [64f6be10-f521-4077-9714-14c3daf0cbdb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -48.63084831048002, -23.491595776006626, -21.626913324799613, -20.64304255626196
LBFGS Accumulation History: 3 points
Removed measurement 7f244c9f to history. Total: 3
th(0)=-48.63084831048002;dx=-3.3167382537675756E-7
New Minimum: -48.63084831048002 > -57.559960471103786
END: th(7.8180126031877E7)=-57.559960471103786; dx=-1.051044138451164E-8 evalInputDelta=8.929112160623767
Fitness changed from -48.63084831048002 to -57.559960471103786
Iteration 7 complete. Error: -57.559960471103786 Total: 1.8457; Orientation: 0.1442; Line Search: 1.1407
Adding measurement 727ac28b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.028e+04, gradient 4.394e-04, dot -0.125; [64f6be10-f521-4077-9714-14c3daf0cbdb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -57.559960471103786, -23.491595776006626, -21.626913324799613, -20.64304255626196
LBFGS Accumulation History: 3 points
Removed measurement 727ac28b to history. Total: 3
th(0)=-57.559960471103786;dx=-1.9098222966943575E-7
New Minimum: -57.559960471103786 > -64.29224297437037
WOLF (strong): th(1.6843397559414053E8)=-64.29224297437037; dx=2.941772039410323E-9 evalInputDelta=6.732282503266582
New Minimum: -64.29224297437037 > -68.0169937540503
END: th(8.421698779707026E7)=-68.0169937540503; dx=-1.3899730675144047E-8 evalInputDelta=10.457033282946519
Fitness changed from -57.559960471103786 to -68.0169937540503
Iteration 8 complete. Error: -68.0169937540503 Total: 2.6519; Orientation: 0.1430; Line Search: 1.9582
Adding measurement 4354892d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.413e+04, gradient 3.130e-04, dot -0.113; [64f6be10-f521-4077-9714-14c3daf0cbdb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -68.0169937540503, -23.491595776006626, -21.626913324799613, -20.64304255626196
LBFGS Accumulation History: 3 points
Removed measurement 4354892d to history. Total: 3
th(0)=-68.0169937540503;dx=-9.689485273529758E-8
New Minimum: -68.0169937540503 > -73.13934037016472
WOLF (strong): th(1.8144000000000003E8)=-73.13934037016472; dx=1.6556523305323122E-9 evalInputDelta=5.122346616114413
New Minimum: -73.13934037016472 > -75.40901253438349
END: th(9.072000000000001E7)=-75.40901253438349; dx=-1.1983920608486722E-8 evalInputDelta=7.3920187803331885
Fitness changed from -68.0169937540503 to -75.40901253438349
Iteration 9 complete. Error: -75.40901253438349 Total: 2.3683; Orientation: 0.1434; Line Search: 1.6672
Adding measurement 22e63403 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.265e+04, gradient 2.503e-04, dot -0.117; [64f6be10-f521-4077-9714-14c3daf0cbdb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -75.40901253438349, -23.491595776006626, -21.626913324799613, -20.64304255626196
LBFGS Accumulation History: 3 points
Removed measurement 22e63403 to history. Total: 3
th(0)=-75.40901253438349;dx=-6.117086720440507E-8
New Minimum: -75.40901253438349 > -76.3023241120262
WOLF (strong): th(1.9545031507969254E8)=-76.3023241120262; dx=1.9775687089513584E-9 evalInputDelta=0.8933115776427059
New Minimum: -76.3023241120262 > -78.7047336254669
END: th(9.772515753984627E7)=-78.7047336254669; dx=-8.0851036694393E-9 evalInputDelta=3.295721091083408
Fitness changed from -75.40901253438349 to -78.7047336254669
Iteration 10 complete. Error: -78.7047336254669 Total: 2.3731; Orientation: 0.1393; Line Search: 1.6871
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 568720b2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.177e+04, gradient 2.081e-04, dot -0.122; [64f6be10-f521-4077-9714-14c3daf0cbdb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -78.7047336254669, -23.491595776006626, -21.626913324799613, -20.64304255626196
LBFGS Accumulation History: 3 points
Removed measurement 568720b2 to history. Total: 3
th(0)=-78.7047336254669;dx=-4.250955821398301E-8
New Minimum: -78.7047336254669 > -79.26545430146929
WOLF (strong): th(2.1054246949267572E8)=-79.26545430146929; dx=1.049009102007208E-9 evalInputDelta=0.56072067600239
New Minimum: -79.26545430146929 > -81.06013999386812
END: th(1.0527123474633786E8)=-81.06013999386812; dx=-5.4148858282031275E-9 evalInputDelta=2.355406368401219
Fitness changed from -78.7047336254669 to -81.06013999386812
Iteration 11 complete. Error: -81.06013999386812 Total: 2.5176; Orientation: 0.1431; Line Search: 1.7266
Adding measurement 4f2df133 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.008e+04, gradient 1.813e-04, dot -0.116; [64f6be10-f521-4077-9714-14c3daf0cbdb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -81.06013999386812, -23.491595776006626, -21.626913324799613, -20.64304255626196
LBFGS Accumulation History: 3 points
Removed measurement 4f2df133 to history. Total: 3
th(0)=-81.06013999386812;dx=-3.221128123346414E-8
Armijo: th(2.268000000000001E8)=-80.36085423283149; dx=1.5112743066404366E-9 evalInputDelta=-0.6992857610366343
New Minimum: -81.06013999386812 > -82.48704265551513
END: th(1.1340000000000004E8)=-82.48704265551513; dx=-3.5641531234721297E-9 evalInputDelta=1.4269026616470057
Fitness changed from -81.06013999386812 to -82.48704265551513
Iteration 12 complete. Error: -82.48704265551513 Total: 2.6335; Orientation: 0.1418; Line Search: 1.9404
Adding measurement 3922bb40 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.970e+03, gradient 1.585e-04, dot -0.123; [64f6be10-f521-4077-9714-14c3daf0cbdb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -82.48704265551513, -23.491595776006626, -21.626913324799613, -20.64304255626196
LBFGS Accumulation History: 3 points
Removed measurement 3922bb40 to history. Total: 3
th(0)=-82.48704265551513;dx=-2.462552957918482E-8
Armijo: th(2.4431289384961572E8)=-81.53578222551685; dx=1.380398691745359E-9 evalInputDelta=-0.9512604299982712
New Minimum: -82.48704265551513 > -83.30672180459085
END: th(1.2215644692480786E8)=-83.30672180459085; dx=-2.4040475583748085E-9 evalInputDelta=0.8196791490757249
Fitness changed from -82.48704265551513 to -83.30672180459085
Iteration 13 complete. Error: -83.30672180459085 Total: 2.4071; Orientation: 0.1406; Line Search: 1.7017
Adding measurement 6cca1b8f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.416e+03, gradient 1.432e-04, dot -0.124; [64f6be10-f521-4077-9714-14c3daf0cbdb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -83.30672180459085, -23.491595776006626, -21.626913324799613, -20.64304255626196
LBFGS Accumulation History: 3 points
Removed measurement 6cca1b8f to history. Total: 3
th(0)=-83.30672180459085;dx=-2.0138626759946018E-8
Armijo: th(2.631780868658447E8)=-82.37602341917052; dx=1.3896247696064024E-9 evalInputDelta=-0.9306983854203281
New Minimum: -83.30672180459085 > -83.96922129967614
END: th(1.3158904343292235E8)=-83.96922129967614; dx=-1.912240543167951E-9 evalInputDelta=0.6624994950852852
Fitness changed from -83.30672180459085 to -83.96922129967614
Iteration 14 complete. Error: -83.96922129967614 Total: 2.3975; Orientation: 0.1401; Line Search: 1.6856
Adding measurement 3bbc9338 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.317e+03, gradient 1.284e-04, dot -0.119; [64f6be10-f521-4077-9714-14c3daf0cbdb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -83.96922129967614, -23.491595776006626, -21.626913324799613, -20.64304255626196
LBFGS Accumulation History: 3 points
Removed measurement 3bbc9338 to history. Total: 3
th(0)=-83.96922129967614;dx=-1.6173751236012165E-8
Armijo: th(2.835000000000002E8)=-82.99838025219836; dx=1.3497065775107327E-9 evalInputDelta=-0.9708410474777764
New Minimum: -83.96922129967614 > -84.39493040253483
END: th(1.417500000000001E8)=-84.39493040253483; dx=-9.687146828578567E-10 evalInputDelta=0.4257091028586899
Fitness changed from -83.96922129967614 to -84.39493040253483
Iteration 15 complete. Error: -84.39493040253483 Total: 2.4541; Orientation: 0.1431; Line Search: 1.7307
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -84.39493040253483 (> -Infinity) after 39.116s (< 5400.000s)
Returns
-84.39493040253483