BasicOptimizer.scala:89 executed in 76.54 seconds (2.042 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 144151127389500
Reset training subject: 144152684372500
Adding measurement 4bac47c6 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-122.58021888827818;dx=-0.015046342346364556
New Minimum: -122.58021888827818 > -122.60969000687923
WOLFE (weak): th(2.154434690031884)=-122.60969000687923; dx=-0.014413545692883552 evalInputDelta=0.02947111860105167
New Minimum: -122.60969000687923 > -122.63915893874379
WOLFE (weak): th(4.308869380063768)=-122.63915893874379; dx=-0.014462161406665817 evalInputDelta=0.05894005046560835
New Minimum: -122.63915893874379 > -122.75706662798291
WOLFE (weak): th(12.926608140191302)=-122.75706662798291; dx=-0.014377719787092761 evalInputDelta=0.17684773970472634
New Minimum: -122.75706662798291 > -123.28777755821277
WOLFE (weak): th(51.70643256076521)=-123.28777755821277; dx=-0.014471448544258778 evalInputDelta=0.7075586699345848
New Minimum: -123.28777755821277 > -126.13304036018513
WOLFE (weak): th(258.53216280382605)=-126.13304036018513; dx=-0.014465269256742972 evalInputDelta=3.5528214719069524
New Minimum: -126.13304036018513 > -143.80932349879657
WOLFE (weak): th(1551.1929768229563)=-143.80932349879657; dx=-0.014331826898637386 evalInputDelta=21.22910461051839
New Minimum: -143.80932349879657 > -257.0733340713109
END: th(10858.350837760694)=-257.0733340713109; dx=-0.009833473672227928 evalInputDelta=134.49311518303273
Fitness changed from -122.58021888827818 to -257.0733340713109
Iteration 1 complete. Error: -257.0733340713109 Total: 10.2739; Orientation: 0.0147; Line Search: 7.0480
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 1e6b073c to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-257.0733340713109;dx=-0.017991833584588744
New Minimum: -257.0733340713109 > -496.0546860566523
END: th(23393.607721408407)=-496.0546860566523; dx=-0.005264234273812793 evalInputDelta=238.9813519853414
Fitness changed from -257.0733340713109 to -496.0546860566523
Iteration 2 complete. Error: -496.0546860566523 Total: 2.8688; Orientation: 0.0139; Line Search: 1.7713
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 6ede5fef to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-496.0546860566523;dx=-0.021492587025563517
New Minimum: -496.0546860566523 > -739.8905074020367
END: th(50400.00000000001)=-739.8905074020367; dx=-0.0018184721201443131 evalInputDelta=243.83582134538443
Fitness changed from -496.0546860566523 to -739.8905074020367
Iteration 3 complete. Error: -739.8905074020367 Total: 2.6516; Orientation: 0.0147; Line Search: 1.7223
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 669268d1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.115e+04, gradient 1.677e-01, dot -0.249; [8ff8446d-86b2-49ae-8944-59b8a27f2580 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -739.8905074020367, -496.0546860566523, -257.0733340713109, -122.58021888827818
LBFGS Accumulation History: 3 points
Removed measurement 669268d1 to history. Total: 3
th(0)=-739.8905074020367;dx=-0.0280942090845005
New Minimum: -739.8905074020367 > -926.7375829731649
WOLF (strong): th(108583.50837760697)=-926.7375829731649; dx=0.0015149559876117633 evalInputDelta=186.84707557112813
New Minimum: -926.7375829731649 > -1117.5585925988842
END: th(54291.75418880348)=-1117.5585925988842; dx=-0.0010898967714661433 evalInputDelta=377.6680851968474
Fitness changed from -739.8905074020367 to -1117.5585925988842
Iteration 4 complete. Error: -1117.5585925988842 Total: 3.9670; Orientation: 0.2984; Line Search: 2.6835
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 7073dc8a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.069e+04, gradient 1.632e-01, dot -0.243; [8ff8446d-86b2-49ae-8944-59b8a27f2580 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1117.5585925988842, -496.0546860566523, -257.0733340713109, -122.58021888827818
LBFGS Accumulation History: 3 points
Removed measurement 7073dc8a to history. Total: 3
th(0)=-1117.5585925988842;dx=-0.026462153338015687
New Minimum: -1117.5585925988842 > -1306.4435684321468
WOLF (strong): th(116968.03860704206)=-1306.4435684321468; dx=0.0024301572945283545 evalInputDelta=188.88497583326262
New Minimum: -1306.4435684321468 > -1609.1807978122704
END: th(58484.01930352103)=-1609.1807978122704; dx=-0.001575893770830348 evalInputDelta=491.6222052133862
Fitness changed from -1117.5585925988842 to -1609.1807978122704
Iteration 5 complete. Error: -1609.1807978122704 Total: 4.1596; Orientation: 0.3272; Line Search: 2.8696
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 3dbdfc2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.794e+04, gradient 1.736e-01, dot -0.231; [8ff8446d-86b2-49ae-8944-59b8a27f2580 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1609.1807978122704, -496.0546860566523, -257.0733340713109, -122.58021888827818
LBFGS Accumulation History: 3 points
Removed measurement 3dbdfc2 to history. Total: 3
th(0)=-1609.1807978122704;dx=-0.030099871473135413
Armijo: th(126000.00000000004)=-1505.3264117460953; dx=0.002788682420932507 evalInputDelta=-103.85438606617504
New Minimum: -1609.1807978122704 > -1913.468550216401
END: th(63000.00000000002)=-1913.468550216401; dx=-1.1916922096105002E-4 evalInputDelta=304.2877524041305
Fitness changed from -1609.1807978122704 to -1913.468550216401
Iteration 6 complete. Error: -1913.468550216401 Total: 3.8571; Orientation: 0.3011; Line Search: 2.6283
Adding measurement 75306493 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.497e+05, gradient 1.780e-01, dot -0.221; [8ff8446d-86b2-49ae-8944-59b8a27f2580 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1913.468550216401, -496.0546860566523, -257.0733340713109, -122.58021888827818
LBFGS Accumulation History: 3 points
Removed measurement 75306493 to history. Total: 3
th(0)=-1913.468550216401;dx=-0.03162208864045751
Armijo: th(135729.38547200873)=-1586.3813777600872; dx=0.003833747705072268 evalInputDelta=-327.08717245631374
New Minimum: -1913.468550216401 > -2179.236790049547
WOLF (strong): th(67864.69273600436)=-2179.236790049547; dx=0.001697318894939516 evalInputDelta=265.768239833146
New Minimum: -2179.236790049547 > -2224.0388494344015
END: th(22621.564245334786)=-2224.0388494344015; dx=-0.007957239466817641 evalInputDelta=310.5702992180006
Fitness changed from -1913.468550216401 to -2224.0388494344015
Iteration 7 complete. Error: -2224.0388494344015 Total: 4.7324; Orientation: 0.2762; Line Search: 3.6234
Adding measurement 15b576e1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.291e+05, gradient 1.614e-01, dot -0.268; [8ff8446d-86b2-49ae-8944-59b8a27f2580 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2224.0388494344015, -496.0546860566523, -257.0733340713109, -122.58021888827818
LBFGS Accumulation History: 3 points
Removed measurement 15b576e1 to history. Total: 3
th(0)=-2224.0388494344015;dx=-0.026020250832463274
New Minimum: -2224.0388494344015 > -2484.225924552343
END: th(48736.6827529342)=-2484.225924552343; dx=-0.003263239697368955 evalInputDelta=260.1870751179413
Fitness changed from -2224.0388494344015 to -2484.225924552343
Iteration 8 complete. Error: -2484.225924552343 Total: 2.9149; Orientation: 0.2674; Line Search: 1.7616
Adding measurement 545f80c5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.128e+08, gradient 1.710e-01, dot -0.274; [8ff8446d-86b2-49ae-8944-59b8a27f2580 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2484.225924552343, -496.0546860566523, -257.0733340713109, -122.58021888827818
LBFGS Accumulation History: 3 points
Removed measurement 545f80c5 to history. Total: 3
th(0)=-2484.225924552343;dx=-0.029204881201364236
Armijo: th(105000.00000000004)=-2435.4469436105237; dx=0.0010699630288964895 evalInputDelta=-48.7789809418191
New Minimum: -2484.225924552343 > -2664.891596018541
END: th(52500.00000000002)=-2664.891596018541; dx=-0.0012329174239511934 evalInputDelta=180.6656714661981
Fitness changed from -2484.225924552343 to -2664.891596018541
Iteration 9 complete. Error: -2664.891596018541 Total: 3.6998; Orientation: 0.2744; Line Search: 2.5920
Adding measurement 551acb57 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.136e+05, gradient 1.732e-01, dot -0.239; [8ff8446d-86b2-49ae-8944-59b8a27f2580 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2664.891596018541, -496.0546860566523, -257.0733340713109, -122.58021888827818
LBFGS Accumulation History: 3 points
Removed measurement 551acb57 to history. Total: 3
th(0)=-2664.891596018541;dx=-0.029937703639899668
Armijo: th(113107.82122667394)=-2370.5668872309257; dx=0.0034021207062404795 evalInputDelta=-294.32470878761524
New Minimum: -2664.891596018541 > -2840.256431361267
WOLF (strong): th(56553.91061333697)=-2840.256431361267; dx=0.0010037424062954345 evalInputDelta=175.3648353427261
New Minimum: -2840.256431361267 > -2864.490032523965
END: th(18851.30353777899)=-2864.490032523965; dx=-0.00711548191567124 evalInputDelta=199.59843650542416
Fitness changed from -2664.891596018541 to -2864.490032523965
Iteration 10 complete. Error: -2864.490032523965 Total: 4.6782; Orientation: 0.2781; Line Search: 3.6061
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement a112366 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 5.770e+04, gradient 1.596e-01, dot 0.383; [8ff8446d-86b2-49ae-8944-59b8a27f2580 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-2864.490032523965;dx=-3523.4481240233526
Armijo: th(40613.90229411183)=-803.1424427738402; dx=0.0319448440444075 evalInputDelta=-2061.347589750125
Armijo: th(20306.951147055915)=-803.328811480955; dx=0.06385313857990904 evalInputDelta=-2061.1612210430103
Armijo: th(6768.9837156853055)=-803.6966862207768; dx=0.1923712811882792 evalInputDelta=-2060.793346303188
Armijo: th(1692.2459289213264)=-804.3365390972187; dx=0.7686339174441508 evalInputDelta=-2060.1534934267465
Armijo: th(338.4491857842653)=-807.2164311722138; dx=3.8544536054602725 evalInputDelta=-2057.2736013517515
Armijo: th(56.40819763071088)=-827.7976703433635; dx=23.008281665801576 evalInputDelta=-2036.6923621806015
Armijo: th(8.058313947244411)=-949.901888467932; dx=155.14155147916227 evalInputDelta=-1914.588144056033
Armijo: th(1.0072892434055514)=-1909.76967133943; dx=770.0146705340501 evalInputDelta=-954.720361184535
Armijo: th(0.11192102704506127)=-2861.2816020792225; dx=34.24562062873616 evalInputDelta=-3.2084304447425893
New Minimum: -2864.490032523965 > -2875.332383973886
END: th(0.011192102704506126)=-2875.332383973886; dx=-992.1358686820506 evalInputDelta=10.842351449920898
Fitness changed from -2864.490032523965 to -2875.332383973886
Iteration 11 complete. Error: -2875.332383973886 Total: 11.4125; Orientation: 0.3034; Line Search: 10.1961
Adding measurement 457f3263 to history. Total: 4
Accepted: LBFGS Orientation magnitude: 2.248e+04, gradient 1.593e-01, dot 0.368; [8ff8446d-86b2-49ae-8944-59b8a27f2580 = 1.000/1.000e+00]
th(0)=-2875.332383973886;dx=-1316.7942307789435
Armijo: th(0.024112654320987664)=-2871.9609684958714; dx=107.4906449138094 evalInputDelta=-3.371415478014569
Armijo: th(0.012056327160493832)=-2873.8533974102834; dx=90.10568435381661 evalInputDelta=-1.4789865636025752
Armijo: th(0.004018775720164611)=-2874.890083032486; dx=72.77376036097176 evalInputDelta=-0.44230094139993525
Armijo: th(0.0010046939300411527)=-2875.2262251240886; dx=68.67946349471276 evalInputDelta=-0.10615884979733892
Armijo: th(2.0093878600823054E-4)=-2875.3114304731125; dx=67.40165092341755 evalInputDelta=-0.020953500773430278
Armijo: th(3.3489797668038423E-5)=-2875.328856099242; dx=67.03068243127059 evalInputDelta=-0.0035278746440781106
Armijo: th(4.784256809719775E-6)=-2875.3317425424543; dx=67.61961303583816 evalInputDelta=-6.41431431631645E-4
Armijo: th(5.980321012149719E-7)=-2875.332383973886; dx=67.3960610537979 evalInputDelta=0.0
Armijo: th(6.644801124610799E-8)=-2875.332383973886; dx=67.18847719541276 evalInputDelta=0.0
New Minimum: -2875.332383973886 > -2875.3324908791105
WOLF (strong): th(6.644801124610799E-9)=-2875.3324908791105; dx=66.93053077433393 evalInputDelta=1.0690522458389751E-4
Armijo: th(6.040728295100727E-10)=-2875.332383973886; dx=67.78558982476201 evalInputDelta=0.0
MIN ALPHA (5.033940245917272E-11): th(6.644801124610799E-9)=-2875.3324908791105
Fitness changed from -2875.332383973886 to -2875.3324908791105
Iteration 12 complete. Error: -2875.3324908791105 Total: 12.4471; Orientation: 0.3689; Line Search: 11.2493
Adding measurement 6f526ddf to history. Total: 5
Rejected: LBFGS Orientation magnitude: 1.636e+07, gradient 1.592e-01, dot -0.218; [8ff8446d-86b2-49ae-8944-59b8a27f2580 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2875.3324908791105, -2875.332383973886, -2864.490032523965, -496.0546860566523, -257.0733340713109, -122.58021888827818
Rejected: LBFGS Orientation magnitude: 2.566e+07, gradient 1.592e-01, dot -0.277; [8ff8446d-86b2-49ae-8944-59b8a27f2580 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2875.3324908791105, -2875.332383973886, -2864.490032523965, -496.0546860566523, -257.0733340713109
Accepted: LBFGS Orientation magnitude: 1.223e+07, gradient 1.592e-01, dot 0.285; [8ff8446d-86b2-49ae-8944-59b8a27f2580 = 1.000/1.000e+00]
Overwriting history with 4 points
th(0)=-2875.3324908791105;dx=-555579.6334572659
Armijo: th(2.1544346900318838E-10)=-2875.3324908791105; dx=-331291.33447889914 evalInputDelta=0.0
Armijo: th(1.0772173450159419E-10)=-2875.332383973886; dx=-329911.29972015193 evalInputDelta=-1.0690522458389751E-4
MIN ALPHA (3.590724483386473E-11): th(0.0)=-2875.3324908791105
Fitness changed from -2875.3324908791105 to -2875.3324908791105
Static Iteration Total: 5.1903; Orientation: 0.9959; Line Search: 3.3755
Iteration 13 failed. Error: -2875.3324908791105
Previous Error: 0.0 -> -2875.3324908791105
Retrying iteration 13
Reset training subject: 144223980625500
Adding measurement 7ca2c201 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-2875.3324908791105;dx=-0.025338026478483017
END: th(2.1544346900318838E-10)=-2875.3324908791105; dx=-0.009021979337765985 evalInputDelta=0.0
Fitness changed from -2875.3324908791105 to -2875.3324908791105
Static Iteration Total: 3.6820; Orientation: 0.0138; Line Search: 1.7797
Iteration 14 failed. Error: -2875.3324908791105
Previous Error: 0.0 -> -2875.3324908791105
Optimization terminated 14
Final threshold in iteration 14: -2875.3324908791105 (> -Infinity) after 76.536s (< 5400.000s)
Returns
-2875.3324908791105