BasicOptimizer.scala:89 executed in 135.36 seconds (1.714 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 72884932357200
Reset training subject: 72887606088400
Adding measurement 2e73875 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-18.807402330552613;dx=-8.507167206652623E-7
New Minimum: -18.807402330552613 > -18.80740438934703
END: th(2.154434690031884)=-18.80740438934703; dx=-7.285003281525727E-7 evalInputDelta=2.05879441850243E-6
Fitness changed from -18.807402330552613 to -18.80740438934703
Iteration 1 complete. Error: -18.80740438934703 Total: 13.1863; Orientation: 0.0596; Line Search: 5.3670
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 56fb3ba6 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-18.80740438934703;dx=-7.259433243013288E-7
New Minimum: -18.80740438934703 > -18.807408506935186
WOLFE (weak): th(4.641588833612779)=-18.807408506935186; dx=-7.25912493607949E-7 evalInputDelta=4.117588154883833E-6
New Minimum: -18.807408506935186 > -18.807411251993457
WOLFE (weak): th(9.283177667225559)=-18.807411251993457; dx=-7.255143742542002E-7 evalInputDelta=6.862646426242236E-6
New Minimum: -18.807411251993457 > -18.807427036070735
WOLFE (weak): th(27.849533001676676)=-18.807427036070735; dx=-7.254465446462374E-7 evalInputDelta=2.264672370344556E-5
New Minimum: -18.807427036070735 > -18.807495662339676
WOLFE (weak): th(111.3981320067067)=-18.807495662339676; dx=-7.252098116606679E-7 evalInputDelta=9.127299264477529E-5
New Minimum: -18.807495662339676 > -18.807861436128963
WOLFE (weak): th(556.9906600335335)=-18.807861436128963; dx=-7.250602148629783E-7 evalInputDelta=4.5704678193203563E-4
New Minimum: -18.807861436128963 > -18.810147875788974
WOLFE (weak): th(3341.9439602012008)=-18.810147875788974; dx=-7.244581311226797E-7 evalInputDelta=0.0027434864419433325
New Minimum: -18.810147875788974 > -18.826572976196147
WOLFE (weak): th(23393.607721408407)=-18.826572976196147; dx=-7.220665992529671E-7 evalInputDelta=0.0191685868491156
New Minimum: -18.826572976196147 > -18.957993873494374
WOLFE (weak): th(187148.86177126726)=-18.957993873494374; dx=-6.93455120166625E-7 evalInputDelta=0.15058948414734274
New Minimum: -18.957993873494374 > -19.77190721331289
END: th(1684339.7559414054)=-19.77190721331289; dx=-2.866442308998705E-7 evalInputDelta=0.9645028239658586
Fitness changed from -18.80740438934703 to -19.77190721331289
Iteration 2 complete. Error: -19.77190721331289 Total: 29.2381; Orientation: 0.0588; Line Search: 26.0059
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 5b46fb59 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-19.77190721331289;dx=-6.577842656057493E-7
New Minimum: -19.77190721331289 > -20.73480082142209
END: th(3628800.0000000005)=-20.73480082142209; dx=-9.506574439971639E-8 evalInputDelta=0.9628936081092014
Fitness changed from -19.77190721331289 to -20.73480082142209
Iteration 3 complete. Error: -20.73480082142209 Total: 8.2688; Orientation: 0.0581; Line Search: 5.1623
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 57edbb30 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.271e+03, gradient 8.339e-04, dot -0.498; [88e93e86-f9b5-4c3f-ac22-3b21b67fad77 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -20.73480082142209, -19.77190721331289, -18.80740438934703, -18.807402330552613
LBFGS Accumulation History: 3 points
Removed measurement 57edbb30 to history. Total: 3
th(0)=-20.73480082142209;dx=-6.948131862328969E-7
New Minimum: -20.73480082142209 > -21.895901264375897
END: th(7818012.603187701)=-21.895901264375897; dx=-1.9003031784052773E-8 evalInputDelta=1.1611004429538063
Fitness changed from -20.73480082142209 to -21.895901264375897
Iteration 4 complete. Error: -21.895901264375897 Total: 9.3267; Orientation: 1.0715; Line Search: 5.2675
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 245dfba to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.811e+03, gradient 7.182e-04, dot -0.225; [88e93e86-f9b5-4c3f-ac22-3b21b67fad77 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -21.895901264375897, -19.77190721331289, -18.80740438934703, -18.807402330552613
LBFGS Accumulation History: 3 points
Removed measurement 245dfba to history. Total: 3
th(0)=-21.895901264375897;dx=-5.149866092635875E-7
New Minimum: -21.895901264375897 > -24.27389047983343
WOLF (strong): th(1.6843397559414055E7)=-24.27389047983343; dx=8.61445408287366E-9 evalInputDelta=2.377989215457532
New Minimum: -24.27389047983343 > -24.362299835839625
END: th(8421698.779707028)=-24.362299835839625; dx=-2.5376406236106658E-8 evalInputDelta=2.4663985714637278
Fitness changed from -21.895901264375897 to -24.362299835839625
Iteration 5 complete. Error: -24.362299835839625 Total: 12.0658; Orientation: 0.9954; Line Search: 7.9127
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 5790f34f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.004e+03, gradient 7.484e-04, dot -0.300; [88e93e86-f9b5-4c3f-ac22-3b21b67fad77 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -24.362299835839625, -19.77190721331289, -18.80740438934703, -18.807402330552613
LBFGS Accumulation History: 3 points
Removed measurement 5790f34f to history. Total: 3
th(0)=-24.362299835839625;dx=-5.598707078732061E-7
New Minimum: -24.362299835839625 > -24.99583405562619
WOLF (strong): th(1.8144000000000004E7)=-24.99583405562619; dx=3.3884483015467516E-8 evalInputDelta=0.633534219786565
New Minimum: -24.99583405562619 > -25.786471627534304
WOLF (strong): th(9072000.000000002)=-25.786471627534304; dx=1.0230195190744487E-7 evalInputDelta=1.424171791694679
New Minimum: -25.786471627534304 > -25.88898527522765
END: th(3024000.0000000005)=-25.88898527522765; dx=-2.58037242822655E-7 evalInputDelta=1.5266854393880251
Fitness changed from -24.362299835839625 to -25.88898527522765
Iteration 6 complete. Error: -25.88898527522765 Total: 14.7671; Orientation: 1.0444; Line Search: 10.5136
Adding measurement 6c13adb8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.330e+04, gradient 6.520e-04, dot -0.466; [88e93e86-f9b5-4c3f-ac22-3b21b67fad77 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -25.88898527522765, -19.77190721331289, -18.80740438934703, -18.807402330552613
LBFGS Accumulation History: 3 points
Removed measurement 6c13adb8 to history. Total: 3
th(0)=-25.88898527522765;dx=-4.245645789292254E-7
New Minimum: -25.88898527522765 > -26.65443356070138
WOLF (strong): th(6515010.502656418)=-26.65443356070138; dx=4.427059765685762E-8 evalInputDelta=0.765448285473731
New Minimum: -26.65443356070138 > -26.741471137646545
END: th(3257505.251328209)=-26.741471137646545; dx=-4.64567718255515E-8 evalInputDelta=0.8524858624188951
Fitness changed from -25.88898527522765 to -26.741471137646545
Iteration 7 complete. Error: -26.741471137646545 Total: 11.2956; Orientation: 1.0058; Line Search: 7.8784
Adding measurement 14205d8f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.526e+04, gradient 6.678e-04, dot -0.457; [88e93e86-f9b5-4c3f-ac22-3b21b67fad77 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -26.741471137646545, -19.77190721331289, -18.80740438934703, -18.807402330552613
LBFGS Accumulation History: 3 points
Removed measurement 14205d8f to history. Total: 3
th(0)=-26.741471137646545;dx=-4.4593070184110867E-7
New Minimum: -26.741471137646545 > -26.95295311338529
WOLF (strong): th(7018082.316422524)=-26.95295311338529; dx=4.28772678151329E-8 evalInputDelta=0.21148197573874583
New Minimum: -26.95295311338529 > -27.258275668225924
WOLF (strong): th(3509041.158211262)=-27.258275668225924; dx=8.500140938821977E-8 evalInputDelta=0.5168045305793783
END: th(1169680.3860704207)=-27.223632104685745; dx=-2.4692270923290174E-7 evalInputDelta=0.4821609670391993
Fitness changed from -26.741471137646545 to -27.258275668225924
Iteration 8 complete. Error: -27.258275668225924 Total: 14.1067; Orientation: 0.9545; Line Search: 10.3457
Adding measurement 54d54928 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.612e+04, gradient 7.160e-04, dot -0.412; [88e93e86-f9b5-4c3f-ac22-3b21b67fad77 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -27.258275668225924, -19.77190721331289, -18.80740438934703, -18.807402330552613
LBFGS Accumulation History: 3 points
Removed measurement 54d54928 to history. Total: 3
th(0)=-27.258275668225924;dx=-5.121851892287301E-7
New Minimum: -27.258275668225924 > -28.06591048454229
WOLF (strong): th(2520000.000000001)=-28.06591048454229; dx=5.9546581598300504E-8 evalInputDelta=0.8076348163163658
END: th(1260000.0000000005)=-27.88932241410044; dx=-3.3002593730625337E-7 evalInputDelta=0.6310467458745173
Fitness changed from -27.258275668225924 to -28.06591048454229
Iteration 9 complete. Error: -28.06591048454229 Total: 11.2972; Orientation: 0.9889; Line Search: 7.8864
Adding measurement 59808f4d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.948e+04, gradient 6.314e-04, dot -0.489; [88e93e86-f9b5-4c3f-ac22-3b21b67fad77 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -28.06591048454229, -19.77190721331289, -18.80740438934703, -18.807402330552613
LBFGS Accumulation History: 3 points
Removed measurement 59808f4d to history. Total: 3
th(0)=-28.06591048454229;dx=-3.9860428778840595E-7
New Minimum: -28.06591048454229 > -28.382161055068572
WOLF (strong): th(2714587.7094401745)=-28.382161055068572; dx=1.0210277627456953E-7 evalInputDelta=0.31625057052628236
New Minimum: -28.382161055068572 > -28.4473014507308
END: th(1357293.8547200873)=-28.4473014507308; dx=-6.949860464444376E-8 evalInputDelta=0.381390966188512
Fitness changed from -28.06591048454229 to -28.4473014507308
Iteration 10 complete. Error: -28.4473014507308 Total: 11.2319; Orientation: 0.9892; Line Search: 7.6979
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -28.4473014507308 (> -Infinity) after 135.364s (< 5400.000s)
Returns
-28.4473014507308