BasicOptimizer.scala:89 executed in 61.08 seconds (0.874 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 147837311296700
Reset training subject: 147838258500200
Adding measurement 7b21078c to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-1.085372339371307;dx=-4.620758128282902E-4
New Minimum: -1.085372339371307 > -1.0866174610262438
WOLFE (weak): th(2.154434690031884)=-1.0866174610262438; dx=-4.6146760514015545E-4 evalInputDelta=0.0012451216549367317
New Minimum: -1.0866174610262438 > -1.0878636653041442
WOLFE (weak): th(4.308869380063768)=-1.0878636653041442; dx=-4.6297556448671726E-4 evalInputDelta=0.002491325932837185
New Minimum: -1.0878636653041442 > -1.0928586502912712
WOLFE (weak): th(12.926608140191302)=-1.0928586502912712; dx=-4.625487726915961E-4 evalInputDelta=0.007486310919964101
New Minimum: -1.0928586502912712 > -1.1155198371022728
WOLFE (weak): th(51.70643256076521)=-1.1155198371022728; dx=-4.6837302515554747E-4 evalInputDelta=0.0301474977309657
New Minimum: -1.1155198371022728 > -1.2429449794529275
WOLFE (weak): th(258.53216280382605)=-1.2429449794529275; dx=-4.949387170238615E-4 evalInputDelta=0.15757264008162042
New Minimum: -1.2429449794529275 > -2.1974838645577544
WOLFE (weak): th(1551.1929768229563)=-2.1974838645577544; dx=-5.829545748766058E-4 evalInputDelta=1.1121115251864473
New Minimum: -2.1974838645577544 > -7.72277791420951
END: th(10858.350837760694)=-7.72277791420951; dx=-2.3857122464023497E-4 evalInputDelta=6.637405574838203
Fitness changed from -1.085372339371307 to -7.72277791420951
Iteration 1 complete. Error: -7.72277791420951 Total: 9.7901; Orientation: 0.0158; Line Search: 7.1815
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 3e840571 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-7.72277791420951;dx=-0.001060612769533107
New Minimum: -7.72277791420951 > -21.21868237601332
END: th(23393.607721408407)=-21.21868237601332; dx=-6.276858819718212E-5 evalInputDelta=13.49590446180381
Fitness changed from -7.72277791420951 to -21.21868237601332
Iteration 2 complete. Error: -21.21868237601332 Total: 2.6731; Orientation: 0.0148; Line Search: 1.6690
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 62d7a7e7 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-21.21868237601332;dx=-0.001975415147293702
New Minimum: -21.21868237601332 > -26.821745091103598
WOLF (strong): th(50400.00000000001)=-26.821745091103598; dx=1.0038864500890471E-4 evalInputDelta=5.603062715090278
New Minimum: -26.821745091103598 > -33.18333867468855
WOLF (strong): th(25200.000000000004)=-33.18333867468855; dx=1.5377351698842766E-4 evalInputDelta=11.96465629867523
New Minimum: -33.18333867468855 > -33.80568224800503
END: th(8400.000000000002)=-33.80568224800503; dx=-3.042157957250538E-4 evalInputDelta=12.586999871991708
Fitness changed from -21.21868237601332 to -33.80568224800503
Iteration 3 complete. Error: -33.80568224800503 Total: 4.3169; Orientation: 0.0167; Line Search: 3.3730
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 51c05993 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.993e+03, gradient 3.580e-02, dot -0.222; [3de3e72f-4c47-4814-83ae-367c4927c365 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -33.80568224800503, -21.21868237601332, -7.72277791420951, -1.085372339371307
LBFGS Accumulation History: 3 points
Removed measurement 51c05993 to history. Total: 3
th(0)=-33.80568224800503;dx=-0.0012813089623979384
New Minimum: -33.80568224800503 > -46.381558412845955
END: th(18097.251396267828)=-46.381558412845955; dx=-1.9011186390959292E-4 evalInputDelta=12.575876164840928
Fitness changed from -33.80568224800503 to -46.381558412845955
Iteration 4 complete. Error: -46.381558412845955 Total: 3.1762; Orientation: 0.2773; Line Search: 1.6973
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 3e3e43c6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.585e+03, gradient 4.184e-02, dot -0.234; [3de3e72f-4c47-4814-83ae-367c4927c365 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -46.381558412845955, -21.21868237601332, -7.72277791420951, -1.085372339371307
LBFGS Accumulation History: 3 points
Removed measurement 3e3e43c6 to history. Total: 3
th(0)=-46.381558412845955;dx=-0.0017485692268334257
New Minimum: -46.381558412845955 > -56.24904469575524
WOLF (strong): th(38989.346202347355)=-56.24904469575524; dx=8.533171281626679E-5 evalInputDelta=9.867486282909283
New Minimum: -56.24904469575524 > -59.77054540154728
END: th(19494.673101173677)=-59.77054540154728; dx=-1.054335999499732E-5 evalInputDelta=13.388986988701326
Fitness changed from -46.381558412845955 to -59.77054540154728
Iteration 5 complete. Error: -59.77054540154728 Total: 3.8555; Orientation: 0.2737; Line Search: 2.6278
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 4e6c2148 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.915e+03, gradient 4.220e-02, dot -0.223; [3de3e72f-4c47-4814-83ae-367c4927c365 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -59.77054540154728, -21.21868237601332, -7.72277791420951, -1.085372339371307
LBFGS Accumulation History: 3 points
Removed measurement 4e6c2148 to history. Total: 3
th(0)=-59.77054540154728;dx=-0.0017672815802533045
New Minimum: -59.77054540154728 > -66.0835549470556
WOLF (strong): th(42000.000000000015)=-66.0835549470556; dx=1.889396737245699E-4 evalInputDelta=6.313009545508322
New Minimum: -66.0835549470556 > -75.8070392069896
WOLF (strong): th(21000.000000000007)=-75.8070392069896; dx=2.569679904155109E-5 evalInputDelta=16.03649380544232
END: th(7000.000000000003)=-72.43351245076333; dx=-7.869801877264267E-4 evalInputDelta=12.66296704921605
Fitness changed from -59.77054540154728 to -75.8070392069896
Iteration 6 complete. Error: -75.8070392069896 Total: 4.7068; Orientation: 0.2731; Line Search: 3.5207
Adding measurement 1f12cdd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.205e+04, gradient 4.172e-02, dot -0.246; [3de3e72f-4c47-4814-83ae-367c4927c365 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -75.8070392069896, -21.21868237601332, -7.72277791420951, -1.085372339371307
LBFGS Accumulation History: 3 points
Removed measurement 1f12cdd to history. Total: 3
th(0)=-75.8070392069896;dx=-0.0017393703966590078
New Minimum: -75.8070392069896 > -92.47015316032655
END: th(15081.042830223192)=-92.47015316032655; dx=-2.2057143569875417E-4 evalInputDelta=16.663113953336946
Fitness changed from -75.8070392069896 to -92.47015316032655
Iteration 7 complete. Error: -92.47015316032655 Total: 2.9206; Orientation: 0.2924; Line Search: 1.8177
Adding measurement 3bb7d59e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.156e+04, gradient 3.254e-02, dot -0.345; [3de3e72f-4c47-4814-83ae-367c4927c365 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -92.47015316032655, -21.21868237601332, -7.72277791420951, -1.085372339371307
LBFGS Accumulation History: 3 points
Removed measurement 3bb7d59e to history. Total: 3
th(0)=-92.47015316032655;dx=-0.0010581083402804081
New Minimum: -92.47015316032655 > -103.73806813173395
END: th(32491.12183528947)=-103.73806813173395; dx=-1.0989990184182213E-6 evalInputDelta=11.267914971407407
Fitness changed from -92.47015316032655 to -103.73806813173395
Iteration 8 complete. Error: -103.73806813173395 Total: 2.8315; Orientation: 0.2722; Line Search: 1.7101
Adding measurement 5ede30dc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.118e+04, gradient 4.159e-02, dot -0.281; [3de3e72f-4c47-4814-83ae-367c4927c365 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -103.73806813173395, -21.21868237601332, -7.72277791420951, -1.085372339371307
LBFGS Accumulation History: 3 points
Removed measurement 5ede30dc to history. Total: 3
th(0)=-103.73806813173395;dx=-0.0017292433079551853
Armijo: th(70000.00000000004)=-88.84383275792867; dx=2.740536909541435E-4 evalInputDelta=-14.894235373805287
New Minimum: -103.73806813173395 > -113.71224243362812
WOLF (strong): th(35000.00000000002)=-113.71224243362812; dx=2.6585974680864814E-4 evalInputDelta=9.974174301894166
New Minimum: -113.71224243362812 > -119.45415063719294
END: th(11666.666666666673)=-119.45415063719294; dx=-3.865386419705685E-4 evalInputDelta=15.716082505458985
Fitness changed from -103.73806813173395 to -119.45415063719294
Iteration 9 complete. Error: -119.45415063719294 Total: 4.8072; Orientation: 0.2751; Line Search: 3.6743
Adding measurement 682bd20c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.638e+04, gradient 2.868e-02, dot -0.411; [3de3e72f-4c47-4814-83ae-367c4927c365 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -119.45415063719294, -21.21868237601332, -7.72277791420951, -1.085372339371307
LBFGS Accumulation History: 3 points
Removed measurement 682bd20c to history. Total: 3
th(0)=-119.45415063719294;dx=-8.151791548805292E-4
New Minimum: -119.45415063719294 > -130.31695944970136
END: th(25135.071383705326)=-130.31695944970136; dx=-2.2839953473033906E-4 evalInputDelta=10.862808812508419
Fitness changed from -119.45415063719294 to -130.31695944970136
Iteration 10 complete. Error: -130.31695944970136 Total: 2.8242; Orientation: 0.2933; Line Search: 1.7263
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 6e80be88 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.591e+05, gradient 3.218e-02, dot -0.401; [3de3e72f-4c47-4814-83ae-367c4927c365 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -130.31695944970136, -21.21868237601332, -7.72277791420951, -1.085372339371307
LBFGS Accumulation History: 3 points
Removed measurement 6e80be88 to history. Total: 3
th(0)=-130.31695944970136;dx=-0.0010342154307377439
New Minimum: -130.31695944970136 > -136.0103525899449
WOLF (strong): th(54151.86972548246)=-136.0103525899449; dx=4.314941173958945E-7 evalInputDelta=5.693393140243529
New Minimum: -136.0103525899449 > -139.51896032966113
END: th(27075.93486274123)=-139.51896032966113; dx=-4.895663530066912E-5 evalInputDelta=9.202000879959769
Fitness changed from -130.31695944970136 to -139.51896032966113
Iteration 11 complete. Error: -139.51896032966113 Total: 3.7910; Orientation: 0.2906; Line Search: 2.5831
Adding measurement 6b65c488 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.306e+05, gradient 3.348e-02, dot -0.359; [3de3e72f-4c47-4814-83ae-367c4927c365 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -139.51896032966113, -21.21868237601332, -7.72277791420951, -1.085372339371307
LBFGS Accumulation History: 3 points
Removed measurement 6b65c488 to history. Total: 3
th(0)=-139.51896032966113;dx=-0.0011180986973184197
Armijo: th(58333.33333333338)=-128.10858799285995; dx=2.569717795921655E-4 evalInputDelta=-11.410372336801174
New Minimum: -139.51896032966113 > -146.76379169935288
WOLF (strong): th(29166.66666666669)=-146.76379169935288; dx=1.1835255975245178E-4 evalInputDelta=7.244831369691752
New Minimum: -146.76379169935288 > -147.82094854738972
END: th(9722.22222222223)=-147.82094854738972; dx=-4.921718450975874E-4 evalInputDelta=8.301988217728592
Fitness changed from -139.51896032966113 to -147.82094854738972
Iteration 12 complete. Error: -147.82094854738972 Total: 4.7632; Orientation: 0.2830; Line Search: 3.6790
Adding measurement 763df5ba to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.172e+07, gradient 2.681e-02, dot -0.476; [3de3e72f-4c47-4814-83ae-367c4927c365 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -147.82094854738972, -21.21868237601332, -7.72277791420951, -1.085372339371307
LBFGS Accumulation History: 3 points
Removed measurement 763df5ba to history. Total: 3
th(0)=-147.82094854738972;dx=-7.181127566927645E-4
New Minimum: -147.82094854738972 > -155.11135328045148
END: th(20945.89281975444)=-155.11135328045148; dx=-2.683839163885324E-4 evalInputDelta=7.290404733061763
Fitness changed from -147.82094854738972 to -155.11135328045148
Iteration 13 complete. Error: -155.11135328045148 Total: 2.8173; Orientation: 0.2864; Line Search: 1.7244
Adding measurement 12d532c3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.620e+05, gradient 2.738e-02, dot -0.469; [3de3e72f-4c47-4814-83ae-367c4927c365 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -155.11135328045148, -21.21868237601332, -7.72277791420951, -1.085372339371307
LBFGS Accumulation History: 3 points
Removed measurement 12d532c3 to history. Total: 3
th(0)=-155.11135328045148;dx=-7.490561529298241E-4
New Minimum: -155.11135328045148 > -163.89681791512717
END: th(45126.55810456872)=-163.89681791512717; dx=-1.3681534182370214E-4 evalInputDelta=8.78546463467569
Fitness changed from -155.11135328045148 to -163.89681791512717
Iteration 14 complete. Error: -163.89681791512717 Total: 2.8279; Orientation: 0.2806; Line Search: 1.7325
Adding measurement 641ac78d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.106e+05, gradient 3.065e-02, dot -0.404; [3de3e72f-4c47-4814-83ae-367c4927c365 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -163.89681791512717, -21.21868237601332, -7.72277791420951, -1.085372339371307
LBFGS Accumulation History: 3 points
Removed measurement 641ac78d to history. Total: 3
th(0)=-163.89681791512717;dx=-9.34453882250783E-4
Armijo: th(97222.2222222223)=-144.53797454642043; dx=1.4530294527877066E-4 evalInputDelta=-19.358843368706744
New Minimum: -163.89681791512717 > -166.34521138622594
WOLF (strong): th(48611.11111111115)=-166.34521138622594; dx=1.3071196936398528E-4 evalInputDelta=2.4483934710987683
New Minimum: -166.34521138622594 > -172.10563217627836
END: th(16203.703703703717)=-172.10563217627836; dx=-2.9726740823799206E-4 evalInputDelta=8.20881426115119
Fitness changed from -163.89681791512717 to -172.10563217627836
Iteration 15 complete. Error: -172.10563217627836 Total: 4.8700; Orientation: 0.2924; Line Search: 3.7208
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -172.10563217627836 (> -Infinity) after 61.080s (< 5400.000s)
Returns
-172.10563217627836