BasicOptimizer.scala:89 executed in 208.65 seconds (2.958 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 19913984643165
Reset training subject: 19917981488213
Adding measurement 1018067e to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.33127665519714355;dx=-5.24986436131473E-7
New Minimum: -0.33127665519714355 > -0.33127710223197937
WOLFE (weak): th(2.154434690031884)=-0.33127710223197937; dx=-4.832079722157982E-7 evalInputDelta=4.470348358154297E-7
New Minimum: -0.33127710223197937 > -0.33127763867378235
WOLFE (weak): th(4.308869380063768)=-0.33127763867378235; dx=-4.832101261169673E-7 evalInputDelta=9.834766387939453E-7
New Minimum: -0.33127763867378235 > -0.33127981424331665
WOLFE (weak): th(12.926608140191302)=-0.33127981424331665; dx=-4.832051993883123E-7 evalInputDelta=3.159046173095703E-6
New Minimum: -0.33127981424331665 > -0.3312892019748688
WOLFE (weak): th(51.70643256076521)=-0.3312892019748688; dx=-4.832058135359623E-7 evalInputDelta=1.2546777725219727E-5
New Minimum: -0.3312892019748688 > -0.33133918046951294
WOLFE (weak): th(258.53216280382605)=-0.33133918046951294; dx=-4.831606250033067E-7 evalInputDelta=6.252527236938477E-5
New Minimum: -0.33133918046951294 > -0.3316514492034912
WOLFE (weak): th(1551.1929768229563)=-0.3316514492034912; dx=-4.830128925447665E-7 evalInputDelta=3.7479400634765625E-4
New Minimum: -0.3316514492034912 > -0.33389705419540405
WOLFE (weak): th(10858.350837760694)=-0.33389705419540405; dx=-4.819911364173718E-7 evalInputDelta=0.002620398998260498
New Minimum: -0.33389705419540405 > -0.3520605266094208
WOLFE (weak): th(86866.80670208555)=-0.3520605266094208; dx=-4.7364154041606113E-7 evalInputDelta=0.02078387141227722
New Minimum: -0.3520605266094208 > -0.5009959936141968
END: th(781801.26031877)=-0.5009959936141968; dx=-3.820076571668069E-7 evalInputDelta=0.16971933841705322
Fitness changed from -0.33127665519714355 to -0.5009959936141968
Iteration 1 complete. Error: -0.5009959936141968 Total: 55.4955; Orientation: 0.0458; Line Search: 43.3518
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 1c7094ed to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-0.5009959936141968;dx=-3.409262436511135E-7
New Minimum: -0.5009959936141968 > -0.7386631965637207
END: th(1684339.7559414052)=-0.7386631965637207; dx=-2.2691076724777497E-7 evalInputDelta=0.23766720294952393
Fitness changed from -0.5009959936141968 to -0.7386631965637207
Iteration 2 complete. Error: -0.7386631965637207 Total: 15.3119; Orientation: 0.0448; Line Search: 10.6559
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 6cc7bb76 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-0.7386631965637207;dx=-2.009583386250767E-7
New Minimum: -0.7386631965637207 > -1.0204138159751892
END: th(3628800.0)=-1.0204138159751892; dx=-1.1481601084329734E-7 evalInputDelta=0.2817506194114685
Fitness changed from -0.7386631965637207 to -1.0204138159751892
Iteration 3 complete. Error: -1.0204138159751892 Total: 13.1385; Orientation: 0.0483; Line Search: 8.6531
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 177cb97a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.169e+03, gradient 3.406e-04, dot -0.838; [234a13e9-2ffb-4253-82b4-348254e8b672 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.0204138159751892, -0.7386631965637207, -0.5009959936141968, -0.33127665519714355
LBFGS Accumulation History: 3 points
Removed measurement 177cb97a to history. Total: 3
th(0)=-1.0204138159751892;dx=-1.1598004534951663E-7
New Minimum: -1.0204138159751892 > -1.314864993095398
END: th(7818012.6031877)=-1.314864993095398; dx=-4.4710048007765754E-8 evalInputDelta=0.29445117712020874
Fitness changed from -1.0204138159751892 to -1.314864993095398
Iteration 4 complete. Error: -1.314864993095398 Total: 13.2812; Orientation: 1.0243; Line Search: 8.4102
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 21157b4e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.741e+03, gradient 2.902e-04, dot -0.617; [234a13e9-2ffb-4253-82b4-348254e8b672 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.314864993095398, -0.7386631965637207, -0.5009959936141968, -0.33127665519714355
LBFGS Accumulation History: 3 points
Removed measurement 21157b4e to history. Total: 3
th(0)=-1.314864993095398;dx=-8.422197243019855E-8
New Minimum: -1.314864993095398 > -1.510568842291832
WOLF (strong): th(1.684339755941405E7)=-1.510568842291832; dx=1.460068934541576E-9 evalInputDelta=0.19570384919643402
END: th(8421698.779707026)=-1.4868974685668945; dx=-1.4390057168756755E-8 evalInputDelta=0.17203247547149658
Fitness changed from -1.314864993095398 to -1.510568842291832
Iteration 5 complete. Error: -1.510568842291832 Total: 19.3812; Orientation: 1.0258; Line Search: 13.8128
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 6826e9d8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.696e+03, gradient 3.171e-04, dot -0.375; [234a13e9-2ffb-4253-82b4-348254e8b672 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.510568842291832, -0.7386631965637207, -0.5009959936141968, -0.33127665519714355
LBFGS Accumulation History: 3 points
Removed measurement 6826e9d8 to history. Total: 3
th(0)=-1.510568842291832;dx=-1.0053250173272898E-7
New Minimum: -1.510568842291832 > -1.6006571501493454
WOLF (strong): th(1.8144E7)=-1.6006571501493454; dx=3.2264660047777306E-8 evalInputDelta=0.09008830785751343
New Minimum: -1.6006571501493454 > -1.6911973506212234
WOLF (strong): th(9072000.0)=-1.6911973506212234; dx=1.3409537558447527E-9 evalInputDelta=0.18062850832939148
END: th(3024000.0)=-1.6264430731534958; dx=-5.249509399195346E-8 evalInputDelta=0.11587423086166382
Fitness changed from -1.510568842291832 to -1.6911973506212234
Iteration 6 complete. Error: -1.6911973506212234 Total: 22.5233; Orientation: 1.0119; Line Search: 17.0966
Adding measurement 38801515 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.266e+03, gradient 2.369e-04, dot -0.390; [234a13e9-2ffb-4253-82b4-348254e8b672 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.6911973506212234, -0.7386631965637207, -0.5009959936141968, -0.33127665519714355
LBFGS Accumulation History: 3 points
Removed measurement 38801515 to history. Total: 3
th(0)=-1.6911973506212234;dx=-5.6111493152362434E-8
New Minimum: -1.6911973506212234 > -1.8007610589265823
END: th(6515010.502656417)=-1.8007610589265823; dx=-1.4466585363541854E-8 evalInputDelta=0.10956370830535889
Fitness changed from -1.6911973506212234 to -1.8007610589265823
Iteration 7 complete. Error: -1.8007610589265823 Total: 15.0080; Orientation: 1.0347; Line Search: 8.9865
Adding measurement 2f83c5db to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.939e+03, gradient 1.779e-04, dot -0.491; [234a13e9-2ffb-4253-82b4-348254e8b672 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.8007610589265823, -0.7386631965637207, -0.5009959936141968, -0.33127665519714355
LBFGS Accumulation History: 3 points
Removed measurement 2f83c5db to history. Total: 3
th(0)=-1.8007610589265823;dx=-3.1645234891682044E-8
New Minimum: -1.8007610589265823 > -1.8846988081932068
WOLF (strong): th(1.4036164632845046E7)=-1.8846988081932068; dx=1.4456135998470157E-9 evalInputDelta=0.08393774926662445
END: th(7018082.316422523)=-1.8708166033029556; dx=-1.0381848560831224E-8 evalInputDelta=0.07005554437637329
Fitness changed from -1.8007610589265823 to -1.8846988081932068
Iteration 8 complete. Error: -1.8846988081932068 Total: 18.1713; Orientation: 1.0525; Line Search: 12.7240
Adding measurement 38703f89 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.601e+03, gradient 2.414e-04, dot -0.336; [234a13e9-2ffb-4253-82b4-348254e8b672 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.8846988081932068, -0.7386631965637207, -0.5009959936141968, -0.33127665519714355
LBFGS Accumulation History: 3 points
Removed measurement 38703f89 to history. Total: 3
th(0)=-1.8846988081932068;dx=-5.8268642160642395E-8
New Minimum: -1.8846988081932068 > -1.89207823574543
WOLF (strong): th(1.5120000000000004E7)=-1.89207823574543; dx=2.183931540746599E-8 evalInputDelta=0.007379427552223206
New Minimum: -1.89207823574543 > -1.9550511091947556
WOLF (strong): th(7560000.000000002)=-1.9550511091947556; dx=7.697762289554022E-9 evalInputDelta=0.07035230100154877
END: th(2520000.0000000005)=-1.9381715208292007; dx=-2.714062409188001E-8 evalInputDelta=0.05347271263599396
Fitness changed from -1.8846988081932068 to -1.9550511091947556
Iteration 9 complete. Error: -1.9550511091947556 Total: 22.0012; Orientation: 1.0020; Line Search: 16.9598
Adding measurement 6edaf379 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.056e+03, gradient 1.893e-04, dot -0.395; [234a13e9-2ffb-4253-82b4-348254e8b672 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.9550511091947556, -0.7386631965637207, -0.5009959936141968, -0.33127665519714355
LBFGS Accumulation History: 3 points
Removed measurement 6edaf379 to history. Total: 3
th(0)=-1.9550511091947556;dx=-3.58463727374698E-8
New Minimum: -1.9550511091947556 > -2.008358135819435
END: th(5429175.418880348)=-2.008358135819435; dx=-5.143119699643303E-9 evalInputDelta=0.053307026624679565
Fitness changed from -1.9550511091947556 to -2.008358135819435
Iteration 10 complete. Error: -2.008358135819435 Total: 14.2658; Orientation: 1.0366; Line Search: 8.4414
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -2.008358135819435 (> -Infinity) after 208.648s (< 5400.000s)
Returns
-2.008358135819435