BasicOptimizer.scala:89 executed in 44.12 seconds (0.751 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 219397986869900
Reset training subject: 219398606712100
Adding measurement 77a04ab4 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-16.43012696315636;dx=-0.00978366470993708
New Minimum: -16.43012696315636 > -16.446516693483648
WOLFE (weak): th(2.154434690031884)=-16.446516693483648; dx=-0.009762241418856316 evalInputDelta=0.016389730327286856
New Minimum: -16.446516693483648 > -16.46289389886574
WOLFE (weak): th(4.308869380063768)=-16.46289389886574; dx=-0.009769402239374054 evalInputDelta=0.03276693570937894
New Minimum: -16.46289389886574 > -16.528646693630076
WOLFE (weak): th(12.926608140191302)=-16.528646693630076; dx=-0.009806730722897769 evalInputDelta=0.09851973047371487
New Minimum: -16.528646693630076 > -16.8212817590012
WOLFE (weak): th(51.70643256076521)=-16.8212817590012; dx=-0.009650446221967273 evalInputDelta=0.39115479584483737
New Minimum: -16.8212817590012 > -18.38320432203459
WOLFE (weak): th(258.53216280382605)=-18.38320432203459; dx=-0.009301563256215239 evalInputDelta=1.9530773588782289
New Minimum: -18.38320432203459 > -26.52623058309784
END: th(1551.1929768229563)=-26.52623058309784; dx=-0.007395894864001425 evalInputDelta=10.096103619941477
Fitness changed from -16.43012696315636 to -26.52623058309784
Iteration 1 complete. Error: -26.52623058309784 Total: 6.0959; Orientation: 0.0166; Line Search: 4.3460
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 35e10b2c to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-26.52623058309784;dx=-0.010583396702526261
New Minimum: -26.52623058309784 > -48.86711083560742
END: th(3341.943960201201)=-48.86711083560742; dx=-0.006283422617000195 evalInputDelta=22.34088025250958
Fitness changed from -26.52623058309784 to -48.86711083560742
Iteration 2 complete. Error: -48.86711083560742 Total: 1.8036; Orientation: 0.0161; Line Search: 1.1637
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 61561678 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-48.86711083560742;dx=-0.010272780617894266
New Minimum: -48.86711083560742 > -79.72963385608381
END: th(7200.000000000001)=-79.72963385608381; dx=-0.0034602115965911328 evalInputDelta=30.862523020476388
Fitness changed from -48.86711083560742 to -79.72963385608381
Iteration 3 complete. Error: -79.72963385608381 Total: 1.7904; Orientation: 0.0161; Line Search: 1.1752
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 4b5eb872 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.741e+03, gradient 9.496e-02, dot -0.342; [ed22fccd-1b06-44f6-afe5-0fe0e4232271 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -79.72963385608381, -48.86711083560742, -26.52623058309784, -16.43012696315636
LBFGS Accumulation History: 3 points
Removed measurement 4b5eb872 to history. Total: 3
th(0)=-79.72963385608381;dx=-0.009015833126145836
New Minimum: -79.72963385608381 > -121.90257616089991
END: th(15511.929768229566)=-121.90257616089991; dx=-0.0017374102957602626 evalInputDelta=42.172942304816104
Fitness changed from -79.72963385608381 to -121.90257616089991
Iteration 4 complete. Error: -121.90257616089991 Total: 2.0837; Orientation: 0.2840; Line Search: 1.2003
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 375d4871 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.161e+03, gradient 9.702e-02, dot -0.197; [ed22fccd-1b06-44f6-afe5-0fe0e4232271 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -121.90257616089991, -48.86711083560742, -26.52623058309784, -16.43012696315636
LBFGS Accumulation History: 3 points
Removed measurement 375d4871 to history. Total: 3
th(0)=-121.90257616089991;dx=-0.009411300351211082
New Minimum: -121.90257616089991 > -172.9609307024637
END: th(33419.43960201202)=-172.9609307024637; dx=-3.836932397565597E-4 evalInputDelta=51.0583545415638
Fitness changed from -121.90257616089991 to -172.9609307024637
Iteration 5 complete. Error: -172.9609307024637 Total: 2.0866; Orientation: 0.2833; Line Search: 1.2328
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 7ede41f6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.270e+03, gradient 9.256e-02, dot -0.138; [ed22fccd-1b06-44f6-afe5-0fe0e4232271 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -172.9609307024637, -48.86711083560742, -26.52623058309784, -16.43012696315636
LBFGS Accumulation History: 3 points
Removed measurement 7ede41f6 to history. Total: 3
th(0)=-172.9609307024637;dx=-0.008559788100464608
New Minimum: -172.9609307024637 > -211.16874006647006
WOLF (strong): th(72000.00000000003)=-211.16874006647006; dx=1.28582644638041E-4 evalInputDelta=38.20780936400635
New Minimum: -211.16874006647006 > -245.06684879785598
END: th(36000.000000000015)=-245.06684879785598; dx=-8.584908388194697E-4 evalInputDelta=72.10591809539227
Fitness changed from -172.9609307024637 to -245.06684879785598
Iteration 6 complete. Error: -245.06684879785598 Total: 2.6602; Orientation: 0.2871; Line Search: 1.7956
Adding measurement 3dec6404 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.593e+03, gradient 8.770e-02, dot -0.137; [ed22fccd-1b06-44f6-afe5-0fe0e4232271 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -245.06684879785598, -48.86711083560742, -26.52623058309784, -16.43012696315636
LBFGS Accumulation History: 3 points
Removed measurement 3dec6404 to history. Total: 3
th(0)=-245.06684879785598;dx=-0.007686481953393789
New Minimum: -245.06684879785598 > -260.79977587872173
WOLF (strong): th(77559.64884114785)=-260.79977587872173; dx=1.4149073139951759E-4 evalInputDelta=15.73292708086575
New Minimum: -260.79977587872173 > -308.0515467019152
END: th(38779.824420573925)=-308.0515467019152; dx=-0.001064233987717472 evalInputDelta=62.9846979040592
Fitness changed from -245.06684879785598 to -308.0515467019152
Iteration 7 complete. Error: -308.0515467019152 Total: 2.9912; Orientation: 0.4999; Line Search: 1.9179
Adding measurement 6e72e55d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.503e+04, gradient 8.897e-02, dot -0.175; [ed22fccd-1b06-44f6-afe5-0fe0e4232271 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -308.0515467019152, -48.86711083560742, -26.52623058309784, -16.43012696315636
LBFGS Accumulation History: 3 points
Removed measurement 6e72e55d to history. Total: 3
th(0)=-308.0515467019152;dx=-0.007914582272732276
Armijo: th(83548.59900503006)=-298.6031012428703; dx=8.061151841149521E-5 evalInputDelta=-9.448445459044876
New Minimum: -308.0515467019152 > -360.8467979750888
END: th(41774.29950251503)=-360.8467979750888; dx=-9.325087329446134E-4 evalInputDelta=52.79525127317362
Fitness changed from -308.0515467019152 to -360.8467979750888
Iteration 8 complete. Error: -360.8467979750888 Total: 2.6035; Orientation: 0.2730; Line Search: 1.7695
Adding measurement 2143937e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.695e+04, gradient 8.991e-02, dot -0.178; [ed22fccd-1b06-44f6-afe5-0fe0e4232271 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -360.8467979750888, -48.86711083560742, -26.52623058309784, -16.43012696315636
LBFGS Accumulation History: 3 points
Removed measurement 2143937e to history. Total: 3
th(0)=-360.8467979750888;dx=-0.00804705619415038
Armijo: th(90000.00000000004)=-342.9590327364144; dx=1.3262158377770348E-4 evalInputDelta=-17.887765238674433
New Minimum: -360.8467979750888 > -414.6322498965174
END: th(45000.00000000002)=-414.6322498965174; dx=-0.0011483022757084954 evalInputDelta=53.7854519214286
Fitness changed from -360.8467979750888 to -414.6322498965174
Iteration 9 complete. Error: -414.6322498965174 Total: 2.5976; Orientation: 0.2826; Line Search: 1.7825
Adding measurement 4cf9eb9b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.088e+06, gradient 9.350e-02, dot -0.193; [ed22fccd-1b06-44f6-afe5-0fe0e4232271 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -414.6322498965174, -48.86711083560742, -26.52623058309784, -16.43012696315636
LBFGS Accumulation History: 3 points
Removed measurement 4cf9eb9b to history. Total: 3
th(0)=-414.6322498965174;dx=-0.008741207209830364
Armijo: th(96949.56105143482)=-382.25288909178573; dx=4.4981146043215496E-4 evalInputDelta=-32.379360804731675
New Minimum: -414.6322498965174 > -471.0834713946935
END: th(48474.78052571741)=-471.0834713946935; dx=-0.0010765783260819532 evalInputDelta=56.451221498176096
Fitness changed from -414.6322498965174 to -471.0834713946935
Iteration 10 complete. Error: -471.0834713946935 Total: 2.6463; Orientation: 0.2814; Line Search: 1.8018
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 69972b37 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 4.328e+04, gradient 9.409e-02, dot 0.197; [ed22fccd-1b06-44f6-afe5-0fe0e4232271 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-471.0834713946935;dx=-804.919250414004
Armijo: th(104435.74875628759)=341.3708863437449; dx=0.00524352240285542 evalInputDelta=-812.4543577384384
Armijo: th(52217.874378143795)=326.5057671407276; dx=0.010232140021212168 evalInputDelta=-797.5892385354211
Armijo: th(17405.95812604793)=301.35861635247795; dx=0.02946290208187736 evalInputDelta=-772.4420877471714
Armijo: th(4351.489531511983)=265.52097907527025; dx=0.1119334552357129 evalInputDelta=-736.6044504699637
Armijo: th(870.2979063023965)=218.49691732403988; dx=0.5257361379652428 evalInputDelta=-689.5803887187334
Armijo: th(145.04965105039943)=147.71616349033064; dx=2.79567940974827 evalInputDelta=-618.7996348850241
Armijo: th(20.72137872148563)=53.268004701969346; dx=15.746052379300062 evalInputDelta=-524.3514760966629
Armijo: th(2.590172340185704)=-122.18815003628134; dx=79.59549159400801 evalInputDelta=-348.89532135841216
Armijo: th(0.28779692668730045)=-440.883845564694; dx=93.72671202399816 evalInputDelta=-30.19962582999949
New Minimum: -471.0834713946935 > -482.8394481994994
END: th(0.028779692668730046)=-482.8394481994994; dx=-292.0333754373272 evalInputDelta=11.755976804805869
Fitness changed from -471.0834713946935 to -482.8394481994994
Iteration 11 complete. Error: -482.8394481994994 Total: 7.8294; Orientation: 0.4931; Line Search: 6.7462
Adding measurement 778259c2 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 4.627e+05, gradient 9.328e-02, dot -0.166; [ed22fccd-1b06-44f6-afe5-0fe0e4232271 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -482.8394481994994, -471.0834713946935, -48.86711083560742, -26.52623058309784, -16.43012696315636
Rejected: LBFGS Orientation magnitude: 6.544e+05, gradient 9.328e-02, dot -0.182; [ed22fccd-1b06-44f6-afe5-0fe0e4232271 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -482.8394481994994, -471.0834713946935, -48.86711083560742, -26.52623058309784
LBFGS Accumulation History: 3 points
Removed measurement 778259c2 to history. Total: 4
Removed measurement 69972b37 to history. Total: 3
th(0)=-482.8394481994994;dx=-0.008699856555052865
New Minimum: -482.8394481994994 > -482.8396742190533
END: th(0.0620039682539683)=-482.8396742190533; dx=-0.007310755201101342 evalInputDelta=2.2601955390655348E-4
Fitness changed from -482.8394481994994 to -482.8396742190533
Iteration 12 complete. Error: -482.8396742190533 Total: 2.3846; Orientation: 0.6313; Line Search: 1.1983
Adding measurement 8006e1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.815e+08, gradient 9.328e-02, dot -0.168; [ed22fccd-1b06-44f6-afe5-0fe0e4232271 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -482.8396742190533, -48.86711083560742, -26.52623058309784, -16.43012696315636
LBFGS Accumulation History: 3 points
Removed measurement 8006e1 to history. Total: 3
th(0)=-482.8396742190533;dx=-0.00869938227848433
New Minimum: -482.8396742190533 > -482.8402359826854
END: th(0.13358350012598497)=-482.8402359826854; dx=-0.007312212617379664 evalInputDelta=5.617636321062491E-4
Fitness changed from -482.8396742190533 to -482.8402359826854
Iteration 13 complete. Error: -482.8402359826854 Total: 2.3140; Orientation: 0.2808; Line Search: 1.4555
Adding measurement 8b217a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.845e+08, gradient 9.328e-02, dot -0.168; [ed22fccd-1b06-44f6-afe5-0fe0e4232271 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -482.8402359826854, -48.86711083560742, -26.52623058309784, -16.43012696315636
LBFGS Accumulation History: 3 points
Removed measurement 8b217a to history. Total: 3
th(0)=-482.8402359826854;dx=-0.008698957737138798
New Minimum: -482.8402359826854 > -482.8415109819139
END: th(0.28779692668730056)=-482.8415109819139; dx=-0.007311586203512448 evalInputDelta=0.0012749992285421285
Fitness changed from -482.8402359826854 to -482.8415109819139
Iteration 14 complete. Error: -482.8415109819139 Total: 2.1069; Orientation: 0.2913; Line Search: 1.2206
Adding measurement 9fbfcf4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.958e+08, gradient 9.328e-02, dot -0.168; [ed22fccd-1b06-44f6-afe5-0fe0e4232271 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -482.8415109819139, -48.86711083560742, -26.52623058309784, -16.43012696315636
LBFGS Accumulation History: 3 points
Removed measurement 9fbfcf4 to history. Total: 3
th(0)=-482.8415109819139;dx=-0.008700228600178496
New Minimum: -482.8415109819139 > -482.8442124436249
END: th(0.6200396825396832)=-482.8442124436249; dx=-0.0073121294170563865 evalInputDelta=0.002701461710955755
Fitness changed from -482.8415109819139 to -482.8442124436249
Iteration 15 complete. Error: -482.8442124436249 Total: 2.1067; Orientation: 0.2870; Line Search: 1.2275
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -482.8442124436249 (> -Infinity) after 44.121s (< 5400.000s)
Returns
-482.8442124436249