BasicOptimizer.scala:89 executed in 46.68 seconds (0.787 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 217153512585600
Reset training subject: 217154134957500
Adding measurement 5eed3c3c to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-143.92334656858142;dx=-0.0603113221541148
New Minimum: -143.92334656858142 > -144.0228590596208
WOLFE (weak): th(2.154434690031884)=-144.0228590596208; dx=-0.0604420176914543 evalInputDelta=0.09951249103937698
New Minimum: -144.0228590596208 > -144.12249897316073
WOLFE (weak): th(4.308869380063768)=-144.12249897316073; dx=-0.060457529541035004 evalInputDelta=0.1991524045793085
New Minimum: -144.12249897316073 > -144.52210450978717
WOLFE (weak): th(12.926608140191302)=-144.52210450978717; dx=-0.060269793879053436 evalInputDelta=0.5987579412057471
New Minimum: -144.52210450978717 > -146.36252915321825
WOLFE (weak): th(51.70643256076521)=-146.36252915321825; dx=-0.06004409431206814 evalInputDelta=2.439182584636825
New Minimum: -146.36252915321825 > -155.98713365911362
WOLFE (weak): th(258.53216280382605)=-155.98713365911362; dx=-0.05597646516230435 evalInputDelta=12.063787090532202
New Minimum: -155.98713365911362 > -196.08164079504783
END: th(1551.1929768229563)=-196.08164079504783; dx=-0.04056716999496658 evalInputDelta=52.15829422646641
Fitness changed from -143.92334656858142 to -196.08164079504783
Iteration 1 complete. Error: -196.08164079504783 Total: 6.2283; Orientation: 0.0159; Line Search: 4.4781
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 3430c6e6 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-196.08164079504783;dx=-0.06992814894257378
New Minimum: -196.08164079504783 > -286.86885363526665
END: th(3341.943960201201)=-286.86885363526665; dx=-0.02707992319336765 evalInputDelta=90.78721284021881
Fitness changed from -196.08164079504783 to -286.86885363526665
Iteration 2 complete. Error: -286.86885363526665 Total: 1.8970; Orientation: 0.0172; Line Search: 1.2704
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 3cc4bf23 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-286.86885363526665;dx=-0.06723083108700212
New Minimum: -286.86885363526665 > -367.2875712886753
END: th(7200.000000000001)=-367.2875712886753; dx=-0.013731307510299075 evalInputDelta=80.41871765340863
Fitness changed from -286.86885363526665 to -367.2875712886753
Iteration 3 complete. Error: -367.2875712886753 Total: 1.8021; Orientation: 0.0165; Line Search: 1.1855
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 7e1deb9c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.984e+03, gradient 2.637e-01, dot -0.207; [804f5adc-cc6c-4a81-b816-4c51e3ec39fd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -367.2875712886753, -286.86885363526665, -196.08164079504783, -143.92334656858142
LBFGS Accumulation History: 3 points
Removed measurement 7e1deb9c to history. Total: 3
th(0)=-367.2875712886753;dx=-0.06953263344961166
New Minimum: -367.2875712886753 > -431.52858741769654
END: th(15511.929768229566)=-431.52858741769654; dx=-0.006088667538038368 evalInputDelta=64.24101612902126
Fitness changed from -367.2875712886753 to -431.52858741769654
Iteration 4 complete. Error: -431.52858741769654 Total: 2.1152; Orientation: 0.2950; Line Search: 1.2523
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 3af799ea to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.607e+03, gradient 2.479e-01, dot -0.144; [804f5adc-cc6c-4a81-b816-4c51e3ec39fd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -431.52858741769654, -286.86885363526665, -196.08164079504783, -143.92334656858142
LBFGS Accumulation History: 3 points
Removed measurement 3af799ea to history. Total: 3
th(0)=-431.52858741769654;dx=-0.06142278261718149
Armijo: th(33419.43960201202)=-429.709770208971; dx=-0.002292698205577255 evalInputDelta=-1.8188172087255339
New Minimum: -431.52858741769654 > -535.6360387063877
END: th(16709.71980100601)=-535.6360387063877; dx=-0.007757372772961724 evalInputDelta=104.10745128869115
Fitness changed from -431.52858741769654 to -535.6360387063877
Iteration 5 complete. Error: -535.6360387063877 Total: 2.6695; Orientation: 0.2914; Line Search: 1.8052
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 7c8709c4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.474e+03, gradient 2.321e-01, dot -0.148; [804f5adc-cc6c-4a81-b816-4c51e3ec39fd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -535.6360387063877, -286.86885363526665, -196.08164079504783, -143.92334656858142
LBFGS Accumulation History: 3 points
Removed measurement 7c8709c4 to history. Total: 3
th(0)=-535.6360387063877;dx=-0.05384682879895665
New Minimum: -535.6360387063877 > -558.8329377283787
END: th(36000.000000000015)=-558.8329377283787; dx=-0.0024456253410800365 evalInputDelta=23.196899021991044
Fitness changed from -535.6360387063877 to -558.8329377283787
Iteration 6 complete. Error: -558.8329377283787 Total: 2.0701; Orientation: 0.2985; Line Search: 1.1907
Adding measurement 5db0c84 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.109e+03, gradient 2.277e-01, dot -0.107; [804f5adc-cc6c-4a81-b816-4c51e3ec39fd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -558.8329377283787, -286.86885363526665, -196.08164079504783, -143.92334656858142
LBFGS Accumulation History: 3 points
Removed measurement 5db0c84 to history. Total: 3
th(0)=-558.8329377283787;dx=-0.051691491868127847
Armijo: th(77559.64884114785)=-498.0036602871345; dx=-2.3826957676860948E-4 evalInputDelta=-60.829277441244244
New Minimum: -558.8329377283787 > -728.1458122054054
END: th(38779.824420573925)=-728.1458122054054; dx=-0.004572447916223179 evalInputDelta=169.31287447702664
Fitness changed from -558.8329377283787 to -728.1458122054054
Iteration 7 complete. Error: -728.1458122054054 Total: 2.9302; Orientation: 0.2784; Line Search: 1.8783
Adding measurement 275092ed to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.593e+04, gradient 2.221e-01, dot -0.114; [804f5adc-cc6c-4a81-b816-4c51e3ec39fd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -728.1458122054054, -286.86885363526665, -196.08164079504783, -143.92334656858142
LBFGS Accumulation History: 3 points
Removed measurement 275092ed to history. Total: 3
th(0)=-728.1458122054054;dx=-0.04932454484289315
Armijo: th(83548.59900503006)=-611.7138248615561; dx=-2.7761395515057475E-4 evalInputDelta=-116.4319873438493
New Minimum: -728.1458122054054 > -900.9293990382695
END: th(41774.29950251503)=-900.9293990382695; dx=-0.004630057986409305 evalInputDelta=172.7835868328641
Fitness changed from -728.1458122054054 to -900.9293990382695
Iteration 8 complete. Error: -900.9293990382695 Total: 2.7511; Orientation: 0.2959; Line Search: 1.8792
Adding measurement 2e3b7f88 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.553e+04, gradient 2.118e-01, dot -0.124; [804f5adc-cc6c-4a81-b816-4c51e3ec39fd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -900.9293990382695, -286.86885363526665, -196.08164079504783, -143.92334656858142
LBFGS Accumulation History: 3 points
Removed measurement 2e3b7f88 to history. Total: 3
th(0)=-900.9293990382695;dx=-0.04479543538415967
Armijo: th(90000.00000000004)=-743.6589881001638; dx=1.1424906953018643E-4 evalInputDelta=-157.27041093810567
New Minimum: -900.9293990382695 > -1088.3291649870523
END: th(45000.00000000002)=-1088.3291649870523; dx=-0.005604704170749762 evalInputDelta=187.39976594878283
Fitness changed from -900.9293990382695 to -1088.3291649870523
Iteration 9 complete. Error: -1088.3291649870523 Total: 2.6981; Orientation: 0.2935; Line Search: 1.8621
Adding measurement 3fd32b6a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.518e+05, gradient 2.170e-01, dot -0.139; [804f5adc-cc6c-4a81-b816-4c51e3ec39fd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1088.3291649870523, -286.86885363526665, -196.08164079504783, -143.92334656858142
LBFGS Accumulation History: 3 points
Removed measurement 3fd32b6a to history. Total: 3
th(0)=-1088.3291649870523;dx=-0.04707754000588407
Armijo: th(96949.56105143482)=-855.851057715539; dx=1.692986636915026E-4 evalInputDelta=-232.4781072715133
New Minimum: -1088.3291649870523 > -1251.822392869972
END: th(48474.78052571741)=-1251.822392869972; dx=-0.005370362308638039 evalInputDelta=163.49322788291965
Fitness changed from -1088.3291649870523 to -1251.822392869972
Iteration 10 complete. Error: -1251.822392869972 Total: 2.6449; Orientation: 0.2901; Line Search: 1.8208
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 3a7321ae to history. Total: 3
Accepted: LBFGS Orientation magnitude: 5.859e+04, gradient 2.178e-01, dot 0.165; [804f5adc-cc6c-4a81-b816-4c51e3ec39fd = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-1251.822392869972;dx=-2110.2596142117036
Armijo: th(104435.74875628759)=179.15586896361958; dx=0.006534278096559051 evalInputDelta=-1430.9782618335917
Armijo: th(52217.874378143795)=178.64370743403003; dx=0.013074061827415665 evalInputDelta=-1430.4661003040019
Armijo: th(17405.95812604793)=177.2637900604941; dx=0.039141516519666936 evalInputDelta=-1429.086182930466
Armijo: th(4351.489531511983)=174.02159568171624; dx=0.15587689065452878 evalInputDelta=-1425.8439885516882
Armijo: th(870.2979063023965)=166.56647469386712; dx=0.7715116351060916 evalInputDelta=-1418.388867563839
Armijo: th(145.04965105039943)=147.8850151135232; dx=4.551187193518794 evalInputDelta=-1399.7074079834952
Armijo: th(20.72137872148563)=64.64161817156275; dx=28.60423542823126 evalInputDelta=-1316.4640110415348
Armijo: th(2.590172340185704)=-330.5511780674459; dx=132.96859111394758 evalInputDelta=-921.2712148025261
Armijo: th(0.28779692668730045)=-1106.594029612549; dx=-36.065807125362085 evalInputDelta=-145.22836325742287
New Minimum: -1251.822392869972 > -1253.2448446499056
END: th(0.028779692668730046)=-1253.2448446499056; dx=-835.0842371611072 evalInputDelta=1.4224517799336809
Fitness changed from -1251.822392869972 to -1253.2448446499056
Iteration 11 complete. Error: -1253.2448446499056 Total: 7.7029; Orientation: 0.2699; Line Search: 6.6201
Adding measurement 6942e43a to history. Total: 4
Rejected: LBFGS Orientation magnitude: 3.373e+05, gradient 2.122e-01, dot -0.130; [804f5adc-cc6c-4a81-b816-4c51e3ec39fd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1253.2448446499056, -1251.822392869972, -286.86885363526665, -196.08164079504783, -143.92334656858142
Rejected: LBFGS Orientation magnitude: 6.325e+05, gradient 2.122e-01, dot -0.134; [804f5adc-cc6c-4a81-b816-4c51e3ec39fd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1253.2448446499056, -1251.822392869972, -286.86885363526665, -196.08164079504783
LBFGS Accumulation History: 3 points
Removed measurement 6942e43a to history. Total: 4
Removed measurement 3a7321ae to history. Total: 3
th(0)=-1253.2448446499056;dx=-0.045011051181181504
New Minimum: -1253.2448446499056 > -1253.2461563424283
END: th(0.0620039682539683)=-1253.2461563424283; dx=-0.037541914395843416 evalInputDelta=0.0013116925226768217
Fitness changed from -1253.2448446499056 to -1253.2461563424283
Iteration 12 complete. Error: -1253.2461563424283 Total: 2.3948; Orientation: 0.6455; Line Search: 1.1919
Adding measurement 548680d4 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 3.867e+04, gradient 2.122e-01, dot 0.176; [804f5adc-cc6c-4a81-b816-4c51e3ec39fd = 1.000/1.000e+00]
th(0)=-1253.2461563424283;dx=-1441.7246132941043
Armijo: th(0.13358350012598497)=-1232.086296142302; dx=-385.1350022460219 evalInputDelta=-21.15986020012633
Armijo: th(0.06679175006299248)=-1247.5224967099564; dx=-511.90930506271263 evalInputDelta=-5.723659632471936
Armijo: th(0.02226391668766416)=-1252.7564593735628; dx=-619.6578054859547 evalInputDelta=-0.48969696886547354
New Minimum: -1253.2461563424283 > -1253.2659513876085
END: th(0.00556597917191604)=-1253.2659513876085; dx=-664.1767340849808 evalInputDelta=0.01979504518021713
Fitness changed from -1253.2461563424283 to -1253.2659513876085
Iteration 13 complete. Error: -1253.2659513876085 Total: 4.2733; Orientation: 0.4271; Line Search: 3.2986
Adding measurement 6bde2361 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.625e+05, gradient 2.115e-01, dot -0.110; [804f5adc-cc6c-4a81-b816-4c51e3ec39fd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1253.2659513876085, -1253.2461563424283, -286.86885363526665, -196.08164079504783, -143.92334656858142
Rejected: LBFGS Orientation magnitude: 6.276e+05, gradient 2.115e-01, dot -0.128; [804f5adc-cc6c-4a81-b816-4c51e3ec39fd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1253.2659513876085, -1253.2461563424283, -286.86885363526665, -196.08164079504783
LBFGS Accumulation History: 3 points
Removed measurement 6bde2361 to history. Total: 4
Removed measurement 548680d4 to history. Total: 3
th(0)=-1253.2659513876085;dx=-0.044690239320118674
New Minimum: -1253.2659513876085 > -1253.2661481627
END: th(0.011991538611970855)=-1253.2661481627; dx=-0.03721471830419447 evalInputDelta=1.9677509135362925E-4
Fitness changed from -1253.2659513876085 to -1253.2661481627
Iteration 14 complete. Error: -1253.2661481627 Total: 2.4261; Orientation: 0.6373; Line Search: 1.2120
Adding measurement 50493b2e to history. Total: 3
Accepted: LBFGS Orientation magnitude: 2.769e+04, gradient 2.114e-01, dot 0.189; [804f5adc-cc6c-4a81-b816-4c51e3ec39fd = 1.000/1.000e+00]
th(0)=-1253.2661481627;dx=-1104.7334115433605
New Minimum: -1253.2661481627 > -1253.5845589010048
END: th(0.025834986772486798)=-1253.5845589010048; dx=-507.25495397717253 evalInputDelta=0.3184107383049195
Fitness changed from -1253.2661481627 to -1253.5845589010048
Iteration 15 complete. Error: -1253.5845589010048 Total: 2.0496; Orientation: 0.2867; Line Search: 1.2296
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -1253.5845589010048 (> -Infinity) after 46.675s (< 5400.000s)
Returns
-1253.5845589010048