BasicOptimizer.scala:89 executed in 40.82 seconds (0.560 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 215952378631100
Reset training subject: 215952997476500
Adding measurement 7bee3e91 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-36.496185448117316;dx=-0.0067550631255273086
New Minimum: -36.496185448117316 > -36.50626702103329
WOLFE (weak): th(2.154434690031884)=-36.50626702103329; dx=-0.006751629542684928 evalInputDelta=0.010081572915972004
New Minimum: -36.50626702103329 > -36.516348192050835
WOLFE (weak): th(4.308869380063768)=-36.516348192050835; dx=-0.006756640893768147 evalInputDelta=0.020162743933518357
New Minimum: -36.516348192050835 > -36.556637627606
WOLFE (weak): th(12.926608140191302)=-36.556637627606; dx=-0.006722874842916504 evalInputDelta=0.060452179488684976
New Minimum: -36.556637627606 > -36.7364692498807
WOLFE (weak): th(51.70643256076521)=-36.7364692498807; dx=-0.006678208040074612 evalInputDelta=0.2402838017633826
New Minimum: -36.7364692498807 > -37.673770728003376
WOLFE (weak): th(258.53216280382605)=-37.673770728003376; dx=-0.006428677947184498 evalInputDelta=1.1775852798860598
New Minimum: -37.673770728003376 > -42.96137889577918
END: th(1551.1929768229563)=-42.96137889577918; dx=-0.005273809750062705 evalInputDelta=6.465193447661861
Fitness changed from -36.496185448117316 to -42.96137889577918
Iteration 1 complete. Error: -42.96137889577918 Total: 5.9792; Orientation: 0.0156; Line Search: 4.2192
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 7b2ae971 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-42.96137889577918;dx=-0.006222395873407999
New Minimum: -42.96137889577918 > -55.323810689815275
END: th(3341.943960201201)=-55.323810689815275; dx=-0.004194308941458848 evalInputDelta=12.362431794036098
Fitness changed from -42.96137889577918 to -55.323810689815275
Iteration 2 complete. Error: -55.323810689815275 Total: 1.8012; Orientation: 0.0160; Line Search: 1.2136
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 6cd6a759 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-55.323810689815275;dx=-0.006059574718127543
New Minimum: -55.323810689815275 > -71.67335413028256
END: th(7200.000000000001)=-71.67335413028256; dx=-0.0024224671351808433 evalInputDelta=16.349543440467286
Fitness changed from -55.323810689815275 to -71.67335413028256
Iteration 3 complete. Error: -71.67335413028256 Total: 2.0608; Orientation: 0.0162; Line Search: 1.4564
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 3e233a56 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.523e+03, gradient 7.630e-02, dot -0.403; [16937b76-7e43-4ff1-b5c4-6778ec16063b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -71.67335413028256, -55.323810689815275, -42.96137889577918, -36.496185448117316
LBFGS Accumulation History: 3 points
Removed measurement 3e233a56 to history. Total: 3
th(0)=-71.67335413028256;dx=-0.005811103097534
New Minimum: -71.67335413028256 > -91.06195715626035
END: th(15511.929768229566)=-91.06195715626035; dx=-0.0013200592992470741 evalInputDelta=19.388603025977787
Fitness changed from -71.67335413028256 to -91.06195715626035
Iteration 4 complete. Error: -91.06195715626035 Total: 2.0404; Orientation: 0.2948; Line Search: 1.1911
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 4e4089d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.939e+03, gradient 7.277e-02, dot -0.288; [16937b76-7e43-4ff1-b5c4-6778ec16063b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -91.06195715626035, -55.323810689815275, -42.96137889577918, -36.496185448117316
LBFGS Accumulation History: 3 points
Removed measurement 4e4089d to history. Total: 3
th(0)=-91.06195715626035;dx=-0.005291194370202482
New Minimum: -91.06195715626035 > -107.3065577391227
END: th(33419.43960201202)=-107.3065577391227; dx=-5.650536780318345E-4 evalInputDelta=16.244600582862347
Fitness changed from -91.06195715626035 to -107.3065577391227
Iteration 5 complete. Error: -107.3065577391227 Total: 2.0775; Orientation: 0.2936; Line Search: 1.1862
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 707930f9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.106e+03, gradient 6.761e-02, dot -0.203; [16937b76-7e43-4ff1-b5c4-6778ec16063b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -107.3065577391227, -55.323810689815275, -42.96137889577918, -36.496185448117316
LBFGS Accumulation History: 3 points
Removed measurement 707930f9 to history. Total: 3
th(0)=-107.3065577391227;dx=-0.004563425345461215
New Minimum: -107.3065577391227 > -118.99632107703228
END: th(72000.00000000003)=-118.99632107703228; dx=-1.4842221675010037E-4 evalInputDelta=11.689763337909582
Fitness changed from -107.3065577391227 to -118.99632107703228
Iteration 6 complete. Error: -118.99632107703228 Total: 2.0795; Orientation: 0.2922; Line Search: 1.2175
Adding measurement 153e8327 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.056e+03, gradient 6.179e-02, dot -0.134; [16937b76-7e43-4ff1-b5c4-6778ec16063b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -118.99632107703228, -55.323810689815275, -42.96137889577918, -36.496185448117316
LBFGS Accumulation History: 3 points
Removed measurement 153e8327 to history. Total: 3
th(0)=-118.99632107703228;dx=-0.0038134245939405098
New Minimum: -118.99632107703228 > -119.53697297804743
WOLF (strong): th(155119.2976822957)=-119.53697297804743; dx=4.07420446530927E-5 evalInputDelta=0.5406519010151527
New Minimum: -119.53697297804743 > -170.44423158267813
END: th(77559.64884114785)=-170.44423158267813; dx=-4.8482168692424933E-4 evalInputDelta=51.44791050564585
Fitness changed from -118.99632107703228 to -170.44423158267813
Iteration 7 complete. Error: -170.44423158267813 Total: 2.7399; Orientation: 0.2912; Line Search: 1.8726
Adding measurement 78c49acc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.020e+03, gradient 6.127e-02, dot -0.155; [16937b76-7e43-4ff1-b5c4-6778ec16063b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -170.44423158267813, -55.323810689815275, -42.96137889577918, -36.496185448117316
LBFGS Accumulation History: 3 points
Removed measurement 78c49acc to history. Total: 3
th(0)=-170.44423158267813;dx=-0.003751424771346261
Armijo: th(167097.19801006012)=-158.60900017717947; dx=3.2301856653726296E-5 evalInputDelta=-11.83523140549866
New Minimum: -170.44423158267813 > -225.81595387993903
END: th(83548.59900503006)=-225.81595387993903; dx=-5.872039797595557E-4 evalInputDelta=55.3717222972609
Fitness changed from -170.44423158267813 to -225.81595387993903
Iteration 8 complete. Error: -225.81595387993903 Total: 2.9043; Orientation: 0.2928; Line Search: 2.0493
Adding measurement 166f1cf9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.124e+04, gradient 6.291e-02, dot -0.170; [16937b76-7e43-4ff1-b5c4-6778ec16063b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -225.81595387993903, -55.323810689815275, -42.96137889577918, -36.496185448117316
LBFGS Accumulation History: 3 points
Removed measurement 166f1cf9 to history. Total: 3
th(0)=-225.81595387993903;dx=-0.003953661134284497
Armijo: th(180000.0000000001)=-189.26858131663866; dx=1.1865619823090212E-4 evalInputDelta=-36.54737256330037
New Minimum: -225.81595387993903 > -278.41621730630663
END: th(90000.00000000004)=-278.41621730630663; dx=-5.515035085869604E-4 evalInputDelta=52.600263426367604
Fitness changed from -225.81595387993903 to -278.41621730630663
Iteration 9 complete. Error: -278.41621730630663 Total: 2.7047; Orientation: 0.2852; Line Search: 1.8554
Adding measurement 66db52f3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.106e+04, gradient 6.538e-02, dot -0.186; [16937b76-7e43-4ff1-b5c4-6778ec16063b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -278.41621730630663, -55.323810689815275, -42.96137889577918, -36.496185448117316
LBFGS Accumulation History: 3 points
Removed measurement 66db52f3 to history. Total: 3
th(0)=-278.41621730630663;dx=-0.004268768669891917
Armijo: th(193899.12210286965)=-220.54890889627274; dx=1.708406008153144E-4 evalInputDelta=-57.8673084100339
New Minimum: -278.41621730630663 > -329.5461449759335
END: th(96949.56105143482)=-329.5461449759335; dx=-5.35054034977356E-4 evalInputDelta=51.12992766962685
Fitness changed from -278.41621730630663 to -329.5461449759335
Iteration 10 complete. Error: -329.5461449759335 Total: 2.6298; Orientation: 0.2875; Line Search: 1.8020
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 6e56be6c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.197e+04, gradient 6.482e-02, dot -0.189; [16937b76-7e43-4ff1-b5c4-6778ec16063b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -329.5461449759335, -55.323810689815275, -42.96137889577918, -36.496185448117316
LBFGS Accumulation History: 3 points
Removed measurement 6e56be6c to history. Total: 3
th(0)=-329.5461449759335;dx=-0.004197318065596839
Armijo: th(208871.49751257518)=-252.35355937466363; dx=1.7377637159271469E-4 evalInputDelta=-77.19258560126985
New Minimum: -329.5461449759335 > -382.7702941390225
END: th(104435.74875628759)=-382.7702941390225; dx=-5.316610951171176E-4 evalInputDelta=53.22414916308901
Fitness changed from -329.5461449759335 to -382.7702941390225
Iteration 11 complete. Error: -382.7702941390225 Total: 2.7106; Orientation: 0.2941; Line Search: 1.8458
Adding measurement 66689097 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.537e+04, gradient 6.593e-02, dot -0.205; [16937b76-7e43-4ff1-b5c4-6778ec16063b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -382.7702941390225, -55.323810689815275, -42.96137889577918, -36.496185448117316
LBFGS Accumulation History: 3 points
Removed measurement 66689097 to history. Total: 3
th(0)=-382.7702941390225;dx=-0.0043406259543508656
Armijo: th(225000.00000000015)=-282.0832735139681; dx=2.392602726418458E-4 evalInputDelta=-100.6870206250544
New Minimum: -382.7702941390225 > -434.22761209411135
END: th(112500.00000000007)=-434.22761209411135; dx=-4.487686785141769E-4 evalInputDelta=51.457317955088854
Fitness changed from -382.7702941390225 to -434.22761209411135
Iteration 12 complete. Error: -434.22761209411135 Total: 2.9388; Orientation: 0.4879; Line Search: 1.8763
Adding measurement 59dfd59b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.236e+05, gradient 6.652e-02, dot -0.199; [16937b76-7e43-4ff1-b5c4-6778ec16063b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -434.22761209411135, -55.323810689815275, -42.96137889577918, -36.496185448117316
LBFGS Accumulation History: 3 points
Removed measurement 59dfd59b to history. Total: 3
th(0)=-434.22761209411135;dx=-0.004420108356959706
Armijo: th(242373.9026285871)=-296.4699477391453; dx=3.3023935834983084E-4 evalInputDelta=-137.75766435496604
New Minimum: -434.22761209411135 > -478.14756046579305
END: th(121186.95131429355)=-478.14756046579305; dx=-3.778031554664825E-4 evalInputDelta=43.9199483716817
Fitness changed from -434.22761209411135 to -478.14756046579305
Iteration 13 complete. Error: -478.14756046579305 Total: 2.7326; Orientation: 0.3180; Line Search: 1.8466
Adding measurement 11fffe3a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.729e+05, gradient 6.734e-02, dot -0.200; [16937b76-7e43-4ff1-b5c4-6778ec16063b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -478.14756046579305, -55.323810689815275, -42.96137889577918, -36.496185448117316
LBFGS Accumulation History: 3 points
Removed measurement 11fffe3a to history. Total: 3
th(0)=-478.14756046579305;dx=-0.004530454097464814
Armijo: th(261089.37189071902)=-322.75298006533393; dx=3.207458576348394E-4 evalInputDelta=-155.39458040045912
New Minimum: -478.14756046579305 > -520.192500973626
END: th(130544.68594535951)=-520.192500973626; dx=-3.725103085572383E-4 evalInputDelta=42.04494050783296
Fitness changed from -478.14756046579305 to -520.192500973626
Iteration 14 complete. Error: -520.192500973626 Total: 2.7445; Orientation: 0.2845; Line Search: 1.8891
Adding measurement 15f2d04a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.661e+04, gradient 6.821e-02, dot -0.155; [16937b76-7e43-4ff1-b5c4-6778ec16063b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -520.192500973626, -55.323810689815275, -42.96137889577918, -36.496185448117316
LBFGS Accumulation History: 3 points
Removed measurement 15f2d04a to history. Total: 3
th(0)=-520.192500973626;dx=-0.004648978364821052
Armijo: th(281250.00000000023)=-330.25544426221853; dx=3.898617246602199E-4 evalInputDelta=-189.93705671140748
New Minimum: -520.192500973626 > -558.4180059237243
END: th(140625.00000000012)=-558.4180059237243; dx=-2.678382975290621E-4 evalInputDelta=38.2255049500983
Fitness changed from -520.192500973626 to -558.4180059237243
Iteration 15 complete. Error: -558.4180059237243 Total: 2.6529; Orientation: 0.2863; Line Search: 1.8307
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -558.4180059237243 (> -Infinity) after 40.820s (< 5400.000s)
Returns
-558.4180059237243