BasicOptimizer.scala:89 executed in 172.34 seconds (2.343 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 4597527735882
Reset training subject: 4598678807172
Adding measurement 51fdc11 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-11.9222652018096;dx=-0.0037482158944587853
New Minimum: -11.9222652018096 > -11.931004781942024
WOLFE (weak): th(2.154434690031884)=-11.931004781942024; dx=-0.003718600763059994 evalInputDelta=0.008739580132424152
New Minimum: -11.931004781942024 > -11.939735453665207
WOLFE (weak): th(4.308869380063768)=-11.939735453665207; dx=-0.0037429248030800767 evalInputDelta=0.0174702518556078
New Minimum: -11.939735453665207 > -11.974702989599347
WOLFE (weak): th(12.926608140191302)=-11.974702989599347; dx=-0.0037380337831828467 evalInputDelta=0.0524377877897475
New Minimum: -11.974702989599347 > -12.132680150980322
WOLFE (weak): th(51.70643256076521)=-12.132680150980322; dx=-0.0037906646398761356 evalInputDelta=0.21041494917072256
New Minimum: -12.132680150980322 > -12.965678218316697
WOLFE (weak): th(258.53216280382605)=-12.965678218316697; dx=-0.0037067923433067966 evalInputDelta=1.0434130165070972
New Minimum: -12.965678218316697 > -17.293578571048275
END: th(1551.1929768229563)=-17.293578571048275; dx=-0.00252328658706648 evalInputDelta=5.371313369238676
Fitness changed from -11.9222652018096 to -17.293578571048275
Iteration 1 complete. Error: -17.293578571048275 Total: 11.7665; Orientation: 0.0179; Line Search: 8.3265
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 2edfb4b9 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-17.293578571048275;dx=-0.003108950811996844
New Minimum: -17.293578571048275 > -25.987227673244952
END: th(3341.943960201201)=-25.987227673244952; dx=-0.0016603908164186513 evalInputDelta=8.693649102196677
Fitness changed from -17.293578571048275 to -25.987227673244952
Iteration 2 complete. Error: -25.987227673244952 Total: 3.6636; Orientation: 0.0177; Line Search: 2.3757
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 42227eef to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-25.987227673244952;dx=-0.003038041352295902
New Minimum: -25.987227673244952 > -36.42515729209866
END: th(7200.000000000001)=-36.42515729209866; dx=-7.713753910633945E-4 evalInputDelta=10.437929618853708
Fitness changed from -25.987227673244952 to -36.42515729209866
Iteration 3 complete. Error: -36.42515729209866 Total: 3.6544; Orientation: 0.0174; Line Search: 2.3629
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 60d4dec9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.721e+03, gradient 7.309e-02, dot -0.319; [8892e067-5f8f-4ce8-b8b7-050f0d773146 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -36.42515729209866, -25.987227673244952, -17.293578571048275, -11.9222652018096
LBFGS Accumulation History: 3 points
Removed measurement 60d4dec9 to history. Total: 3
th(0)=-36.42515729209866;dx=-0.005228248617357208
New Minimum: -36.42515729209866 > -49.97969924644136
END: th(15511.929768229566)=-49.97969924644136; dx=-2.218911770355487E-4 evalInputDelta=13.554541954342703
Fitness changed from -36.42515729209866 to -49.97969924644136
Iteration 4 complete. Error: -49.97969924644136 Total: 4.1145; Orientation: 0.4769; Line Search: 2.3674
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 4f64957d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.810e+03, gradient 8.549e-02, dot -0.188; [8892e067-5f8f-4ce8-b8b7-050f0d773146 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -49.97969924644136, -25.987227673244952, -17.293578571048275, -11.9222652018096
LBFGS Accumulation History: 3 points
Removed measurement 4f64957d to history. Total: 3
th(0)=-49.97969924644136;dx=-0.00725502878898371
New Minimum: -49.97969924644136 > -54.75259931391865
WOLF (strong): th(33419.43960201202)=-54.75259931391865; dx=2.0144716433034295E-4 evalInputDelta=4.772900067477288
New Minimum: -54.75259931391865 > -65.07093835642229
WOLF (strong): th(16709.71980100601)=-65.07093835642229; dx=1.3427048112792675E-4 evalInputDelta=15.091239109980926
New Minimum: -65.07093835642229 > -69.41412875878049
END: th(5569.906600335336)=-69.41412875878049; dx=-0.0013420094275051564 evalInputDelta=19.43442951233913
Fitness changed from -49.97969924644136 to -69.41412875878049
Iteration 5 complete. Error: -69.41412875878049 Total: 6.9036; Orientation: 0.8738; Line Search: 4.7400
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement aac361b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.439e+03, gradient 8.059e-02, dot -0.202; [8892e067-5f8f-4ce8-b8b7-050f0d773146 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -69.41412875878049, -25.987227673244952, -17.293578571048275, -11.9222652018096
LBFGS Accumulation History: 3 points
Removed measurement aac361b to history. Total: 3
th(0)=-69.41412875878049;dx=-0.00637412138108245
New Minimum: -69.41412875878049 > -79.45931779328781
END: th(12000.000000000004)=-79.45931779328781; dx=-1.832614057783948E-4 evalInputDelta=10.045189034507317
Fitness changed from -69.41412875878049 to -79.45931779328781
Iteration 6 complete. Error: -79.45931779328781 Total: 4.0765; Orientation: 0.4614; Line Search: 2.3352
Adding measurement 28da2ccb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.249e+03, gradient 9.690e-02, dot -0.170; [8892e067-5f8f-4ce8-b8b7-050f0d773146 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -79.45931779328781, -25.987227673244952, -17.293578571048275, -11.9222652018096
LBFGS Accumulation History: 3 points
Removed measurement 28da2ccb to history. Total: 3
th(0)=-79.45931779328781;dx=-0.00919713784181921
Armijo: th(25853.216280382614)=-77.18398004074442; dx=5.046117793377318E-4 evalInputDelta=-2.2753377525433933
New Minimum: -79.45931779328781 > -95.73846948314214
WOLF (strong): th(12926.608140191307)=-95.73846948314214; dx=3.7111953296662813E-4 evalInputDelta=16.27915168985433
New Minimum: -95.73846948314214 > -101.95727307929944
END: th(4308.869380063769)=-101.95727307929944; dx=-0.0025194757217574847 evalInputDelta=22.49795528601163
Fitness changed from -79.45931779328781 to -101.95727307929944
Iteration 7 complete. Error: -101.95727307929944 Total: 6.4229; Orientation: 0.4644; Line Search: 4.8250
Adding measurement 14e45593 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.931e+04, gradient 8.417e-02, dot -0.234; [8892e067-5f8f-4ce8-b8b7-050f0d773146 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -101.95727307929944, -25.987227673244952, -17.293578571048275, -11.9222652018096
LBFGS Accumulation History: 3 points
Removed measurement 14e45593 to history. Total: 3
th(0)=-101.95727307929944;dx=-0.007008791155693946
New Minimum: -101.95727307929944 > -116.71540500452289
END: th(9283.177667225562)=-116.71540500452289; dx=-8.729568559391682E-4 evalInputDelta=14.758131925223452
Fitness changed from -101.95727307929944 to -116.71540500452289
Iteration 8 complete. Error: -116.71540500452289 Total: 3.9543; Orientation: 0.4579; Line Search: 2.3542
Adding measurement 58b582b0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.050e+05, gradient 1.065e-01, dot -0.181; [8892e067-5f8f-4ce8-b8b7-050f0d773146 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -116.71540500452289, -25.987227673244952, -17.293578571048275, -11.9222652018096
LBFGS Accumulation History: 3 points
Removed measurement 58b582b0 to history. Total: 3
th(0)=-116.71540500452289;dx=-0.011261566765029917
Armijo: th(20000.00000000001)=-98.69052727861106; dx=8.177958561611568E-4 evalInputDelta=-18.024877725911836
New Minimum: -116.71540500452289 > -125.08840473305646
WOLF (strong): th(10000.000000000005)=-125.08840473305646; dx=7.622270946346312E-4 evalInputDelta=8.372999728533571
New Minimum: -125
...skipping 8981 bytes...
297.60660661610956 > -297.79412392624585
WOLFE (weak): th(30.53337145736799)=-297.79412392624585; dx=-0.009869988702541703 evalInputDelta=0.2501811949911712
New Minimum: -297.79412392624585 > -298.7873933225569
WOLFE (weak): th(152.66685728683996)=-298.7873933225569; dx=-0.009703277048575439 evalInputDelta=1.243450591302235
New Minimum: -298.7873933225569 > -304.5918022020973
END: th(916.0011437210397)=-304.5918022020973; dx=-0.008920253656999715 evalInputDelta=7.0478594708426385
Fitness changed from -297.5439427312547 to -304.5918022020973
Iteration 16 complete. Error: -304.5918022020973 Total: 11.9698; Orientation: 1.0252; Line Search: 9.6772
Adding measurement 596be6c5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.822e+04, gradient 9.489e-02, dot -0.130; [8892e067-5f8f-4ce8-b8b7-050f0d773146 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -304.5918022020973, -25.987227673244952, -17.293578571048275, -11.9222652018096
LBFGS Accumulation History: 3 points
Removed measurement 596be6c5 to history. Total: 3
th(0)=-304.5918022020973;dx=-0.008738264031837984
New Minimum: -304.5918022020973 > -316.3724170181695
END: th(1973.4646401414893)=-316.3724170181695; dx=-0.007196246047924892 evalInputDelta=11.780614816072159
Fitness changed from -304.5918022020973 to -316.3724170181695
Iteration 17 complete. Error: -316.3724170181695 Total: 3.9611; Orientation: 0.4562; Line Search: 2.3875
Adding measurement 7c5f815b to history. Total: 3
Accepted: LBFGS Orientation magnitude: 2.767e+04, gradient 8.865e-02, dot 0.215; [8892e067-5f8f-4ce8-b8b7-050f0d773146 = 1.000/1.000e+00]
th(0)=-316.3724170181695;dx=-526.5614460865831
Armijo: th(4251.700680272113)=-164.28285432381597; dx=0.018857585348183943 evalInputDelta=-152.0895626943535
Armijo: th(2125.8503401360563)=-164.17130317449246; dx=0.03730956504185292 evalInputDelta=-152.201113843677
Armijo: th(708.6167800453521)=-164.41437488472695; dx=0.11109039364563399 evalInputDelta=-151.95804213344252
Armijo: th(177.15419501133803)=-164.07823397057706; dx=0.4421678005796772 evalInputDelta=-152.2941830475924
Armijo: th(35.430839002267604)=-166.5486339117487; dx=2.159537405555798 evalInputDelta=-149.82378310642076
Armijo: th(5.905139833711267)=-170.76474329009773; dx=10.006432590800202 evalInputDelta=-145.60767372807175
Armijo: th(0.8435914048158953)=-250.0338019067888; dx=25.627131139679484 evalInputDelta=-66.33861511138068
New Minimum: -316.3724170181695 > -332.2168258273189
END: th(0.10544892560198692)=-332.2168258273189; dx=-238.02652242339278 evalInputDelta=15.844408809149456
Fitness changed from -316.3724170181695 to -332.2168258273189
Iteration 18 complete. Error: -332.2168258273189 Total: 12.3141; Orientation: 0.4475; Line Search: 10.7298
Adding measurement 25de5758 to history. Total: 4
Accepted: LBFGS Orientation magnitude: 1.452e+04, gradient 9.556e-02, dot 0.146; [8892e067-5f8f-4ce8-b8b7-050f0d773146 = 1.000/1.000e+00]
th(0)=-332.2168258273189;dx=-200.30150010577316
Armijo: th(0.22718282334351186)=-326.2259717351923; dx=-49.050969062588834 evalInputDelta=-5.9908540921266535
Armijo: th(0.11359141167175593)=-330.73470919454934; dx=-72.13676989412073 evalInputDelta=-1.4821166327695892
New Minimum: -332.2168258273189 > -332.2551932667357
END: th(0.03786380389058531)=-332.2551932667357; dx=-91.06035215204584 evalInputDelta=0.03836743941678833
Fitness changed from -332.2168258273189 to -332.2551932667357
Iteration 19 complete. Error: -332.2551932667357 Total: 6.4827; Orientation: 0.5675; Line Search: 4.7772
Adding measurement aae872a to history. Total: 5
Accepted: LBFGS Orientation magnitude: 1.048e+04, gradient 9.691e-02, dot 0.149; [8892e067-5f8f-4ce8-b8b7-050f0d773146 = 1.000/1.000e+00]
th(0)=-332.2551932667357;dx=-151.09276210435252
Armijo: th(0.0815750925984412)=-331.5668170227437; dx=-58.28578135039065 evalInputDelta=-0.6883762439920247
Armijo: th(0.0407875462992206)=-332.03993328621743; dx=-63.093586553627965 evalInputDelta=-0.2152599805182831
Armijo: th(0.013595848766406867)=-332.2165331642259; dx=-66.6146073389452 evalInputDelta=-0.03866010250982299
Armijo: th(0.003398962191601717)=-332.2497564803314; dx=-68.40554069057086 evalInputDelta=-0.005436786404288796
Armijo: th(6.797924383203434E-4)=-332.2543307796456; dx=-69.06481303161937 evalInputDelta=-8.62487090103059E-4
Armijo: th(1.1329873972005723E-4)=-332.2550854559719; dx=-68.70491423992537 evalInputDelta=-1.07810763836369E-4
Armijo: th(1.618553424572246E-5)=-332.2551778652002; dx=-68.75386791735893 evalInputDelta=-1.5401535506498476E-5
New Minimum: -332.2551932667357 > -332.2552086682706
END: th(2.0231917807153077E-6)=-332.2552086682706; dx=-67.82956949161985 evalInputDelta=1.540153488122087E-5
Fitness changed from -332.2551932667357 to -332.2552086682706
Iteration 20 complete. Error: -332.2552086682706 Total: 12.9694; Orientation: 0.6725; Line Search: 11.1786
<a id="p-17"></a>Iteration 20
<a id="p-16"></a>
Adding measurement 47cec92 to history. Total: 6
Rejected: LBFGS Orientation magnitude: 7.902e+03, gradient 9.659e-02, dot -0.012; [8892e067-5f8f-4ce8-b8b7-050f0d773146 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -332.2552086682706, -332.2551932667357, -332.2168258273189, -316.3724170181695, -25.987227673244952, -17.293578571048275, -11.9222652018096
Rejected: LBFGS Orientation magnitude: 3.025e+04, gradient 9.659e-02, dot -0.083; [8892e067-5f8f-4ce8-b8b7-050f0d773146 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -332.2552086682706, -332.2551932667357, -332.2168258273189, -316.3724170181695, -25.987227673244952, -17.293578571048275
Accepted: LBFGS Orientation magnitude: 1.006e+05, gradient 9.659e-02, dot 0.115; [8892e067-5f8f-4ce8-b8b7-050f0d773146 = 1.000/1.000e+00]
Overwriting history with 5 points
th(0)=-332.2552086682706;dx=-1107.2487454086013
Armijo: th(4.358834556960439E-6)=-332.25470041724407; dx=-558.9997543274796 evalInputDelta=-5.082510265310702E-4
Armijo: th(2.1794172784802194E-6)=-332.25493144053434; dx=-563.4228113389286 evalInputDelta=-2.772277362623754E-4
Armijo: th(7.264724261600731E-7)=-332.25511625905085; dx=-563.7291331885001 evalInputDelta=-9.240921974651428E-5
Armijo: th(1.8161810654001828E-7)=-332.2551778652002; dx=-547.0221644690134 evalInputDelta=-3.0803070387719345E-5
Armijo: th(3.6323621308003656E-8)=-332.2551932667357; dx=-562.8615367919047 evalInputDelta=-1.540153488122087E-5
Armijo: th(6.053936884667276E-9)=-332.2552086682706; dx=-560.0215170498675 evalInputDelta=0.0
Armijo: th(8.648481263810393E-10)=-332.2552086682706; dx=-553.7209824327742 evalInputDelta=0.0
Armijo: th(1.0810601579762992E-10)=-332.2552086682706; dx=-557.6821635206575 evalInputDelta=0.0
MIN ALPHA (1.2011779533069991E-11): th(0.0)=-332.2552086682706
Fitness changed from -332.2552086682706 to -332.2552086682706
Static Iteration Total: 15.4951; Orientation: 2.2255; Line Search: 12.0019
Iteration 21 failed. Error: -332.2552086682706
Previous Error: 0.0 -> -332.2552086682706
Retrying iteration 21
Reset training subject: 4759263023609
Adding measurement 7267360c to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-332.2552086682706;dx=-0.00936792615275847
WOLFE (weak): th(2.1544346900318838E-10)=-332.2552086682706; dx=-0.008721727817054964 evalInputDelta=0.0
WOLFE (weak): th(4.3088693800637676E-10)=-332.2552086682706; dx=-0.008749404977082473 evalInputDelta=0.0
WOLFE (weak): th(1.2926608140191302E-9)=-332.2552086682706; dx=-0.008724311898383913 evalInputDelta=0.0
WOLFE (weak): th(5.170643256076521E-9)=-332.2552086682706; dx=-0.00873185476286394 evalInputDelta=0.0
WOLFE (weak): th(2.5853216280382606E-8)=-332.2552086682706; dx=-0.008713089178802695 evalInputDelta=0.0
END: th(1.5511929768229565E-7)=-332.2552086682706; dx=-0.008362073650147338 evalInputDelta=0.0
Fitness changed from -332.2552086682706 to -332.2552086682706
Static Iteration Total: 10.6087; Orientation: 0.0163; Line Search: 8.3348
Iteration 22 failed. Error: -332.2552086682706
Previous Error: 0.0 -> -332.2552086682706
Optimization terminated 22
Final threshold in iteration 22: -332.2552086682706 (> -Infinity) after 172.345s (< 3600.000s)
Returns
-332.2552086682706