BasicOptimizer.scala:89 executed in 73.69 seconds (0.791 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 157647181937700
Reset training subject: 157648062512600
Adding measurement 290b0961 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-66.5637243894147;dx=-0.008996039328771075
New Minimum: -66.5637243894147 > -66.5894860762689
WOLFE (weak): th(2.154434690031884)=-66.5894860762689; dx=-0.008657688178894587 evalInputDelta=0.025761686854195887
New Minimum: -66.5894860762689 > -66.61526685999789
WOLFE (weak): th(4.308869380063768)=-66.61526685999789; dx=-0.00864136576589098 evalInputDelta=0.05154247058318617
New Minimum: -66.61526685999789 > -66.71838902453987
WOLFE (weak): th(12.926608140191302)=-66.71838902453987; dx=-0.008664744858881898 evalInputDelta=0.15466463512517237
New Minimum: -66.71838902453987 > -67.18383678860398
WOLFE (weak): th(51.70643256076521)=-67.18383678860398; dx=-0.008675357366415455 evalInputDelta=0.6201123991892814
New Minimum: -67.18383678860398 > -69.7044486992432
WOLFE (weak): th(258.53216280382605)=-69.7044486992432; dx=-0.008855322571646585 evalInputDelta=3.1407243098284994
New Minimum: -69.7044486992432 > -86.54304701524548
WOLFE (weak): th(1551.1929768229563)=-86.54304701524548; dx=-0.009421910612598464 evalInputDelta=19.979322625830775
New Minimum: -86.54304701524548 > -201.306363138902
END: th(10858.350837760694)=-201.306363138902; dx=-0.005423639005051403 evalInputDelta=134.7426387494873
Fitness changed from -66.5637243894147 to -201.306363138902
Iteration 1 complete. Error: -201.306363138902 Total: 9.3306; Orientation: 0.0155; Line Search: 6.8607
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 453edc9c to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-201.306363138902;dx=-0.013064515753578242
New Minimum: -201.306363138902 > -370.68276817267383
END: th(23393.607721408407)=-370.68276817267383; dx=-0.001476026046611502 evalInputDelta=169.37640503377185
Fitness changed from -201.306363138902 to -370.68276817267383
Iteration 2 complete. Error: -370.68276817267383 Total: 2.7210; Orientation: 0.0148; Line Search: 1.7230
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 3681089e to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-370.68276817267383;dx=-0.016723428032536434
New Minimum: -370.68276817267383 > -469.03984203582
WOLF (strong): th(50400.00000000001)=-469.03984203582; dx=7.236422251431547E-4 evalInputDelta=98.35707386314618
New Minimum: -469.03984203582 > -534.9584552838609
END: th(25200.000000000004)=-534.9584552838609; dx=-3.35591558341988E-5 evalInputDelta=164.27568711118704
Fitness changed from -370.68276817267383 to -534.9584552838609
Iteration 3 complete. Error: -534.9584552838609 Total: 3.5022; Orientation: 0.0145; Line Search: 2.5948
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 74f70e61 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.054e+04, gradient 1.378e-01, dot -0.324; [41470b67-9541-49a7-8d2b-12b4c9926e25 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -534.9584552838609, -370.68276817267383, -201.306363138902, -66.5637243894147
LBFGS Accumulation History: 3 points
Removed measurement 74f70e61 to history. Total: 3
th(0)=-534.9584552838609;dx=-0.01899049162663602
New Minimum: -534.9584552838609 > -570.5748277035486
WOLF (strong): th(54291.75418880348)=-570.5748277035486; dx=0.0022970533625254644 evalInputDelta=35.616372419687764
New Minimum: -570.5748277035486 > -690.2441677780531
WOLF (strong): th(27145.87709440174)=-690.2441677780531; dx=5.534590235421227E-4 evalInputDelta=155.28571249419224
END: th(9048.625698133914)=-665.287674600923; dx=-0.005842186783244354 evalInputDelta=130.3292193170621
Fitness changed from -534.9584552838609 to -690.2441677780531
Iteration 4 complete. Error: -690.2441677780531 Total: 4.8722; Orientation: 0.2783; Line Search: 3.6137
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 851ef54 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.141e+04, gradient 1.312e-01, dot -0.272; [41470b67-9541-49a7-8d2b-12b4c9926e25 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -690.2441677780531, -370.68276817267383, -201.306363138902, -66.5637243894147
LBFGS Accumulation History: 3 points
Removed measurement 851ef54 to history. Total: 3
th(0)=-690.2441677780531;dx=-0.017169139660631015
New Minimum: -690.2441677780531 > -845.8180265267474
END: th(19494.673101173677)=-845.8180265267474; dx=-0.0018217789124041436 evalInputDelta=155.5738587486943
Fitness changed from -690.2441677780531 to -845.8180265267474
Iteration 5 complete. Error: -845.8180265267474 Total: 2.9385; Orientation: 0.2762; Line Search: 1.7321
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 65ebc59e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.508e+04, gradient 1.162e-01, dot -0.349; [41470b67-9541-49a7-8d2b-12b4c9926e25 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -845.8180265267474, -370.68276817267383, -201.306363138902, -66.5637243894147
LBFGS Accumulation History: 3 points
Removed measurement 65ebc59e to history. Total: 3
th(0)=-845.8180265267474;dx=-0.013441810040970312
New Minimum: -845.8180265267474 > -910.8211105365581
WOLF (strong): th(42000.000000000015)=-910.8211105365581; dx=2.756696359218376E-4 evalInputDelta=65.00308400981066
New Minimum: -910.8211105365581 > -936.6192852115016
END: th(21000.000000000007)=-936.6192852115016; dx=-0.0017080262430838496 evalInputDelta=90.8012586847542
Fitness changed from -845.8180265267474 to -936.6192852115016
Iteration 6 complete. Error: -936.6192852115016 Total: 3.7233; Orientation: 0.2761; Line Search: 2.5236
Adding measurement 605b5960 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.880e+04, gradient 1.185e-01, dot -0.381; [41470b67-9541-49a7-8d2b-12b4c9926e25 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -936.6192852115016, -370.68276817267383, -201.306363138902, -66.5637243894147
LBFGS Accumulation History: 3 points
Removed measurement 605b5960 to history. Total: 3
th(0)=-936.6192852115016;dx=-0.014004401465068945
New Minimum: -936.6192852115016 > -997.4732056037655
WOLF (strong): th(45243.12849066957)=-997.4732056037655; dx=6.65674664190529E-4 evalInputDelta=60.85392039226383
New Minimum: -997.4732056037655 > -1032.4061964936063
END: th(22621.564245334786)=-1032.4061964936063; dx=-0.0012021497706591565 evalInputDelta=95.78691128210471
Fitness changed from -936.6192852115016 to -1032.4061964936063
Iteration 7 complete. Error: -1032.4061964936063 Total: 3.9271; Orientation: 0.2766; Line Search: 2.5919
Adding measurement cc8669c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.807e+04, gradient 1.198e-01, dot -0.386; [41470b67-9541-49a7-8d2b-12b4c9926e25 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1032.4061964936063, -370.68276817267383, -201.306363138902, -66.5637243894147
LBFGS Accumulation History: 3 points
Removed measurement cc8669c to history. Total: 3
th(0)=-1032.4061964936063;dx=-0.014327638390165752
New Minimum: -1032.4061964936063 > -1039.200053045486
WOLF (strong): th(48736.6827529342)=-1039.200053045486; dx=0.0013931092799392719 evalInputDelta=6.793856551879571
New Minimum: -1039.200053045486 > -1116.2537739716656
END: th(24368.3413764671)=-1116.2537739716656; dx=-6.328547696863099E-4 evalInputDelta=83.84757747805929
Fitness changed from -1032.4061964936063 to -1116.2537739716656
Iteration 8 complete. Error: -1116.2537739716656 Total: 3.6557; Orientation: 0.2765; Line Search: 2.5559
Adding measurement 69a71e22 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.358e+05, gradient 1.216e-01, dot -0.388; [41470b67-9541-49a7-8d2b-12b4c9926e25 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1116.2537739716656, -370.68276817267383, -201.306363138902, -66.5637243894147
LBFGS Accumulation History: 3 points
Removed measurement 69a71e22 to history. Total: 3
th(0)=-1116.2537739716656;dx=-0.01475511988565731
Armijo: th(52500.00000000002)=-1084.9403172227885; dx=0.002151517765365054 evalInputDelta=-31.313456748877115
New Minimum: -1116.2537739716656 > -1203.1036907703558
WOLF (strong): th(26250.00000000001)=-1203.1036907703558; dx=1.7875371416644357E-4 evalInputDelta=86.84991679869017
END: th(8750.000000000004)=-1186.193629872739; dx=-0.004542412350043962 evalInputDelta=69.93985590107332
Fitness changed from -1116.2537739716656 to -1203.1036907703558
Iteration 9 complete. Error: -1203.1036907703558 Total: 4.7392; Orientation: 0.2827; Line Search: 3.6619
Adding measurement f8d4dd3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.629e+05, gradient 1.244e-01, dot -0.380; [41470b67-9541-49a7-8d2b-12b4c9926e25 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1203.1036907703558, -370.68276817267383, -201.306363138902, -66.5637243894147
LBFGS Accumulation History: 3 points
Removed measurement f8d4dd3 to history. Total: 3
th(0)=-1203.1036907703558;dx=-0.015408706854152894
New Minimum: -1203.1036907703558 > -1293.4184804666545
END: th(18851.303537778993)=-1293.4184804666545; dx=-0.0010268120967920813 evalInputDelta=90.3147896962987
Fitness changed from -1203.1036907703558 to -1293.4184804666545
Iteration 10 complete. Error: -1293.4184804666545 Total: 2.7885; Orientation: 0.2792; Line Search: 1.7067
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 323bd797 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.722e+05, gradient 1.128e-01, dot -0.432; [41470b67-9541-49a7-8d2b-12b4c9926e25 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1293.4184804666545, -370.68276817267383, -201.306363138902, -66.5637243894147
LBFGS Accumulation History: 3 points
Removed measurement 323bd797 to history. Total: 3
th(0)=-1293.4184804666545;dx=-0.012684293894475073
New Minimum: -1293.4184804666545 > -1344.1761508015777
END: th(40613.90229411184)=-1344.1761508015777; dx=-2.5073164072617423E-4 evalInputDelta=50.757670334923205
Fitness changed from -1293.4184804666545 to -1344.1761508015777
Iteration 11 complete. Error: -1344.1761508015777 Total: 2.9307; Orientation: 0.2753; Line Search: 1.7377
Adding measurement 7d75d02 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.939e+06, gradient 1.300e-01, dot -0.372; [41470b67-9541-49a7-8d2b-12b4c9926e25 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1344.1761508015777, -370.68276817267383, -201.306363138902, -66.5637243894147
LBFGS Accumulation History: 3 points
Removed measurement 7d75d02 to history. Total: 3
th(0)=-1344.1761508015777;dx=-0.01686422745520061
Armijo: th(87500.00000000006)=-1080.4003580629133; dx=0.002737158703360212 evalInputDelta=-263.77579273866445
New Minimum: -1344.1761508015777 > -1366.8421366407592
WOLF (strong): th(43750.00000000003)=-1366.8421366407592; dx=0.002448793160659568 evalInputDelta=22.66598583918153
New Minimum: -1366.8421366407592 > -1457.1288731647574
END: th(14583.333333333343)=-1457.1288731647574; dx=-0.0026460383170825085 evalInputDelta=112.95272236317965
Fitness changed from -1344.1761508015777 to -1457.1288731647574
Iteration 12 complete. Error: -1457.1288731647574 Total: 4.7478; Orientation: 0.2721; Line Search: 3.6139
Adding measurement 16b7ac to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.327e+06, gradient 1.092e-01, dot -0.431; [41470b67-9541-49a7-8d2b-12b4c9926e25 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1457.1288731647574, -370.68276817267383, -201.306363138902, -66.5637243894147
LBFGS Accumulation History: 3 points
Removed measurement 16b7ac to history. Total: 3
th(0)=-1457.1288731647574;dx=-0.011893078358816672
New Minimum: -1457.1288731647574 > -1538.4376080000634
END: th(31418.83922963166)=-1538.4376080000634; dx=-0.0020704302822281122 evalInputDelta=81.30873483530604
Fitness changed from -1457.1288731647574 to -1538.4376080000634
Iteration 13 complete. Error: -1538.4376080000634 Total: 2.7992; Orientation: 0.2873; Line Search: 1.7261
Adding measurement 6a1294c0 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.604e+04, gradient 1.101e-01, dot 0.160; [41470b67-9541-49a7-8d2b-12b4c9926e25 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-1538.4376080000634;dx=-280.8471496344948
Armijo: th(67689.83715685307)=-99.35585409130097; dx=0.002290679108759612 evalInputDelta=-1439.0817539087625
Armijo: th(33844.918578426536)=-99.32158435725827; dx=0.004596179405290452 evalInputDelta=-1439.1160236428052
Armijo: th(11281.639526142179)=-99.08133307449891; dx=0.013918527598443047 evalInputDelta=-1439.3562749255645
Armijo: th(2820.4098815355446)=-98.6044928938777; dx=0.055410700731731044 evalInputDelta=-1439.8331151061857
Armijo: th(564.0819763071089)=-98.66381878945964; dx=0.27716599537722764 evalInputDelta=-1439.7737892106038
Armijo: th(94.01366271785149)=-100.20838680972723; dx=1.6834981535501239 evalInputDelta=-1438.2292211903361
Armijo: th(13.430523245407356)=-144.15339303933007; dx=16.5681450026348 evalInputDelta=-1394.2842149607334
Armijo: th(1.6788154056759195)=-692.8573293241456; dx=188.11721508450432 evalInputDelta=-845.5802786759178
Armijo: th(0.18653504507510216)=-1441.7208880523697; dx=350.61606769088667 evalInputDelta=-96.71671994769372
Armijo: th(0.018653504507510216)=-1536.0714669545278; dx=-46.33821933842715 evalInputDelta=-2.3661410455356418
Armijo: th(0.0016957731370463832)=-1538.3198351704211; dx=-112.456558613668 evalInputDelta=-0.11777282964226288
Armijo: th(1.413144280871986E-4)=-1538.4283952427193; dx=-118.81051436782113 evalInputDelta=-0.009212757344130296
Armijo: th(1.08703406220922E-5)=-1538.4368349036656; dx=-117.93078672486082 evalInputDelta=-7.730963977792271E-4
Armijo: th(7.764529015780144E-7)=-1538.4375435753784; dx=-119.22719043829494 evalInputDelta=-6.442468497880327E-5
Armijo: th(5.176352677186762E-8)=-1538.4376080000634; dx=-118.00028106504523 evalInputDelta=0.0
Armijo: th(3.2352204232417263E-9)=-1538.4375435753784; dx=-117.83036273042926 evalInputDelta=-6.442468497880327E-5
Armijo: th(1.9030708372010156E-10)=-1538.4375435753784; dx=-118.71755810923801 evalInputDelta=-6.442468497880327E-5
MIN ALPHA (1.0572615762227863E-11): th(0.0)=-1538.4376080000634
Fitness changed from -1538.4376080000634 to -1538.4376080000634
Static Iteration Total: 17.6317; Orientation: 0.2796; Line Search: 16.5135
Iteration 14 failed. Error: -1538.4376080000634
Previous Error: 0.0 -> -1538.4376080000634
Retrying iteration 14
Reset training subject: 157717489577500
Adding measurement 5ec4d86d to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-1538.4376080000634;dx=-0.012099010078310133
END: th(2.1544346900318838E-10)=-1538.4376080000634; dx=-0.004133526191905717 evalInputDelta=0.0
Fitness changed from -1538.4376080000634 to -1538.4376080000634
Static Iteration Total: 3.3847; Orientation: 0.0145; Line Search: 1.7224
Iteration 15 failed. Error: -1538.4376080000634
Previous Error: 0.0 -> -1538.4376080000634
Optimization terminated 15
Final threshold in iteration 15: -1538.4376080000634 (> -Infinity) after 73.692s (< 5400.000s)
Returns
-1538.4376080000634