BasicOptimizer.scala:89 executed in 9.81 seconds (0.677 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 161586483346100
Reset training subject: 161586615106500
Adding measurement 29ca3cda to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-33.031423692778894;dx=-0.2758428147365975
New Minimum: -33.031423692778894 > -33.79090222592743
WOLFE (weak): th(2.154434690031884)=-33.79090222592743; dx=-0.27942079244730367 evalInputDelta=0.7594785331485383
New Minimum: -33.79090222592743 > -34.56370611311
WOLFE (weak): th(4.308869380063768)=-34.56370611311; dx=-0.27909206792152563 evalInputDelta=1.5322824203311072
New Minimum: -34.56370611311 > -37.76361444792957
WOLFE (weak): th(12.926608140191302)=-37.76361444792957; dx=-0.298509106215278 evalInputDelta=4.732190755150675
New Minimum: -37.76361444792957 > -54.35332946405514
WOLFE (weak): th(51.70643256076521)=-54.35332946405514; dx=-0.337726286680769 evalInputDelta=21.32190577127625
New Minimum: -54.35332946405514 > -131.85364904175006
END: th(258.53216280382605)=-131.85364904175006; dx=-0.2065218321974285 evalInputDelta=98.82222534897116
Fitness changed from -33.031423692778894 to -131.85364904175006
Iteration 1 complete. Error: -131.85364904175006 Total: 1.0305; Orientation: 0.0016; Line Search: 0.6787
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 7a10f50d to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-131.85364904175006;dx=-0.6577587748241632
New Minimum: -131.85364904175006 > -245.10292799658677
END: th(556.9906600335335)=-245.10292799658677; dx=-0.04255620194391139 evalInputDelta=113.24927895483671
Fitness changed from -131.85364904175006 to -245.10292799658677
Iteration 2 complete. Error: -245.10292799658677 Total: 0.3457; Orientation: 0.0015; Line Search: 0.2267
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 76176715 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-245.10292799658677;dx=-0.6813474640852362
New Minimum: -245.10292799658677 > -276.0132752695058
WOLF (strong): th(1200.0)=-276.0132752695058; dx=0.02061183637711058 evalInputDelta=30.91034727291904
New Minimum: -276.0132752695058 > -361.8847041013201
WOLF (strong): th(600.0)=-361.8847041013201; dx=0.008211182977709293 evalInputDelta=116.78177610473332
New Minimum: -361.8847041013201 > -363.37848295544416
END: th(200.0)=-363.37848295544416; dx=-0.23515202683251513 evalInputDelta=118.27555495885738
Fitness changed from -245.10292799658677 to -363.37848295544416
Iteration 3 complete. Error: -363.37848295544416 Total: 0.5745; Orientation: 0.0015; Line Search: 0.4523
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 7a689949 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.845e+02, gradient 6.250e-01, dot -0.135; [7afdbe76-bfde-4d2e-af29-681e534c7657 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -363.37848295544416, -245.10292799658677, -131.85364904175006, -33.031423692778894
LBFGS Accumulation History: 3 points
Removed measurement 7a689949 to history. Total: 3
th(0)=-363.37848295544416;dx=-0.38065477496865874
New Minimum: -363.37848295544416 > -452.86789257464477
END: th(430.88693800637674)=-452.86789257464477; dx=-0.10810758057490377 evalInputDelta=89.48940961920061
Fitness changed from -363.37848295544416 to -452.86789257464477
Iteration 4 complete. Error: -452.86789257464477 Total: 0.3783; Orientation: 0.0280; Line Search: 0.2319
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 7c405bca to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.367e+03, gradient 6.364e-01, dot -0.232; [7afdbe76-bfde-4d2e-af29-681e534c7657 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -452.86789257464477, -245.10292799658677, -131.85364904175006, -33.031423692778894
LBFGS Accumulation History: 3 points
Removed measurement 7c405bca to history. Total: 3
th(0)=-452.86789257464477;dx=-0.39616005087769024
New Minimum: -452.86789257464477 > -457.86066726134055
WOLF (strong): th(928.3177667225558)=-457.86066726134055; dx=0.0035917373451880077 evalInputDelta=4.992774686695782
New Minimum: -457.86066726134055 > -510.416389455707
END: th(464.1588833612779)=-510.416389455707; dx=-0.08367802498857188 evalInputDelta=57.54849688106225
Fitness changed from -452.86789257464477 to -510.416389455707
Iteration 5 complete. Error: -510.416389455707 Total: 0.4957; Orientation: 0.0278; Line Search: 0.3462
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 37a125fd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.387e+03, gradient 6.398e-01, dot -0.165; [7afdbe76-bfde-4d2e-af29-681e534c7657 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -510.416389455707, -245.10292799658677, -131.85364904175006, -33.031423692778894
LBFGS Accumulation History: 3 points
Removed measurement 37a125fd to history. Total: 3
th(0)=-510.416389455707;dx=-0.40175756469479973
New Minimum: -510.416389455707 > -545.7225141956304
END: th(1000.0000000000001)=-545.7225141956304; dx=-0.03442887304911393 evalInputDelta=35.306124739923405
Fitness changed from -510.416389455707 to -545.7225141956304
Iteration 6 complete. Error: -545.7225141956304 Total: 0.3747; Orientation: 0.0266; Line Search: 0.2258
Adding measurement 4e6df9e2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.068e+03, gradient 9.226e-01, dot -0.246; [7afdbe76-bfde-4d2e-af29-681e534c7657 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -545.7225141956304, -245.10292799658677, -131.85364904175006, -33.031423692778894
LBFGS Accumulation History: 3 points
Removed measurement 4e6df9e2 to history. Total: 3
th(0)=-545.7225141956304;dx=-0.8430676486697434
Armijo: th(2154.434690031884)=-315.8272124516822; dx=0.05120050374514952 evalInputDelta=-229.8953017439482
New Minimum: -545.7225141956304 > -547.0069126619882
WOLF (strong): th(1077.217345015942)=-547.0069126619882; dx=0.11273676876340902 evalInputDelta=1.284398466357743
New Minimum: -547.0069126619882 > -718.4889091002032
END: th(359.07244833864735)=-718.4889091002032; dx=-0.13239375055291358 evalInputDelta=172.76639490457273
Fitness changed from -545.7225141956304 to -718.4889091002032
Iteration 7 complete. Error: -718.4889091002032 Total: 0.6172; Orientation: 0.0311; Line Search: 0.4516
Adding measurement 24936074 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.295e+03, gradient 6.935e-01, dot -0.194; [7afdbe76-bfde-4d2e-af29-681e534c7657 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -718.4889091002032, -245.10292799658677, -131.85364904175006, -33.031423692778894
LBFGS Accumulation History: 3 points
Removed measurement 24936074 to history. Total: 3
th(0)=-718.4889091002032;dx=-0.4686012871855064
New Minimum: -718.4889091002032 > -845.3403900204935
END: th(773.5981389354633)=-845.3403900204935; dx=-0.035300830508953024 evalInputDelta=126.85148092029033
Fitness changed from -718.4889091002032 to -845.3403900204935
Iteration 8 complete. Error: -845.3403900204935 Total: 0.3623; Orientation: 0.0262; Line Search: 0.2255
Adding measurement 287cb447 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.466e+04, gradient 7.502e-01, dot -0.275; [7afdbe76-bfde-4d2e-af29-681e534c7657 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -845.3403900204935, -245.10292799658677, -131.85364904175006, -33.031423692778894
LBFGS Accumulation History: 3 points
Removed measurement 287cb447 to history. Total: 3
th(0)=-845.3403900204935;dx=-0.5403028555087008
New Minimum: -845.3403900204935 > -864.9100025436172
WOLF (strong): th(1666.6666666666672)=-864.9100025436172; dx=0.03379276185568371 evalInputDelta=19.569612523123737
New Minimum: -864.9100025436172 > -997.4064805785051
END: th(833.3333333333336)=-997.4064805785051; dx=-0.03713548312219018 evalInputDelta=152.06609055801164
Fitness changed from -845.3403900204935 to -997.4064805785051
Iteration 9 complete. Error: -997.4064805785051 Total: 0.4761; Orientation: 0.0276; Line Search: 0.3414
Adding measurement 4642412c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.972e+04, gradient 7.075e-01, dot -0.273; [7afdbe76-bfde-4d2e-af29-681e534c7657 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -997.4064805785051, -245.10292799658677, -131.85364904175006, -33.031423692778894
LBFGS Accumulation History: 3 points
Removed measurement 4642412c to history. Total: 3
th(0)=-997.4064805785051;dx=-0.49303062235019524
Armijo: th(1795.3622416932371)=-774.7126080037681; dx=0.08554290733216241 evalInputDelta=-222.69387257473704
New Minimum: -997.4064805785051 > -1055.5562514617588
WOLF (strong): th(897.6811208466186)=-1055.5562514617588; dx=0.03163185174205886 evalInputDelta=58.14977088325372
New Minimum: -1055.5562514617588 > -1101.0715235623888
END: th(299.2270402822062)=-1101.0715235623888; dx=-0.24105676641432427 evalInputDelta=103.66504298388372
Fitness changed from -997.4064805785051 to -1101.0715235623888
Iteration 10 complete. Error: -1101.0715235623888 Total: 0.5841; Orientation: 0.0282; Line Search: 0.4470
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 6cb81df1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.524e+04, gradient 5.640e-01, dot -0.377; [7afdbe76-bfde-4d2e-af29-681e534c7657 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1101.0715235623888, -245.10292799658677, -131.85364904175006, -33.031423692778894
LBFGS Accumulation History: 3 points
Removed measurement 6cb81df1 to history. Total: 3
th(0)=-1101.0715235623888;dx=-0.3017662673070296
New Minimum: -1101.0715235623888 > -1179.6014157332977
END: th(644.6651157795529)=-1179.6014157332977; dx=-0.1297319963288549 evalInputDelta=78.52989217090885
Fitness changed from -1101.0715235623888 to -1179.6014157332977
Iteration 11 complete. Error: -1179.6014157332977 Total: 0.3783; Orientation: 0.0266; Line Search: 0.2295
Adding measurement 6df88876 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 2.084e+04, gradient 5.983e-01, dot 0.418; [7afdbe76-bfde-4d2e-af29-681e534c7657 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-1179.6014157332977;dx=-5213.111397743121
Armijo: th(1388.8888888888896)=-308.2778145399049; dx=0.14541659584987948 evalInputDelta=-871.3236011933927
Armijo: th(694.4444444444448)=-306.98844644383604; dx=0.2932612170469293 evalInputDelta=-872.6129692894617
Armijo: th(231.4814814814816)=-308.828380739206; dx=0.8701409145405747 evalInputDelta=-870.7730349940916
Armijo: th(57.8703703703704)=-310.34519591094045; dx=3.7443104276843777 evalInputDelta=-869.2562198223573
Armijo: th(11.57407407407408)=-304.95377467331014; dx=17.144529843261658 evalInputDelta=-874.6476410599876
Armijo: th(1.9290123456790134)=-343.8961393902235; dx=58.172722091406996 evalInputDelta=-835.7052763430743
Armijo: th(0.27557319223985904)=-788.6755907849564; dx=-254.74239872592068 evalInputDelta=-390.9258249483413
New Minimum: -1179.6014157332977 > -1240.8775120857015
END: th(0.03444664902998238)=-1240.8775120857015; dx=-1775.654071532896 evalInputDelta=61.27609635240378
Fitness changed from -1179.6014157332977 to -1240.8775120857015
Iteration 12 complete. Error: -1240.8775120857015 Total: 1.1471; Orientation: 0.0269; Line Search: 1.0098
Adding measurement 7de0969e to history. Total: 4
Accepted: LBFGS Orientation magnitude: 1.189e+04, gradient 7.448e-01, dot 0.079; [7afdbe76-bfde-4d2e-af29-681e534c7657 = 1.000/1.000e+00]
th(0)=-1240.8775120857015;dx=-712.6212261837971
Armijo: th(0.07421305562554718)=-1090.4779915248175; dx=437.4048752742923 evalInputDelta=-150.39952056088396
Armijo: th(0.03710652781277359)=-1177.9870542582375; dx=129.13291605072556 evalInputDelta=-62.89045782746393
Armijo: th(0.012368842604257864)=-1223.7760211738093; dx=-49.9572067371011 evalInputDelta=-17.101490911892142
Armijo: th(0.003092210651064466)=-1237.010660827141; dx=-272.55686827116443 evalInputDelta=-3.866851258560473
Armijo: th(6.184421302128932E-4)=-1240.1383995345036; dx=-256.8081924793103 evalInputDelta=-0.7391125511978771
Armijo: th(1.0307368836881553E-4)=-1240.7554150597127; dx=-292.9980588723243 evalInputDelta=-0.12209702598875083
Armijo: th(1.4724812624116504E-5)=-1240.8601351481964; dx=-298.69289647093376 evalInputDelta=-0.01737693750510516
Armijo: th(1.840601578014563E-6)=-1240.8752959101087; dx=-278.7828084040153 evalInputDelta=-0.002216175592820946
Armijo: th(2.0451128644606256E-7)=-1240.877109144979; dx=-328.71862642616145 evalInputDelta=-4.029407225516479E-4
Armijo: th(2.0451128644606257E-8)=-1240.877360982946; dx=-303.35970849632105 evalInputDelta=-1.5110275558072317E-4
New Minimum: -1240.8775120857015 > -1240.8776128208615
END: th(1.8591935131460235E-9)=-1240.8776128208615; dx=-307.01687699567464 evalInputDelta=1.0073516000375093E-4
Fitness changed from -1240.8775120857015 to -1240.8776128208615
Iteration 13 complete. Error: -1240.8776128208615 Total: 2.1874; Orientation: 0.0335; Line Search: 2.0454
Adding measurement 54c6a50d to history. Total: 5
Rejected: LBFGS Orientation magnitude: 4.103e+05, gradient 7.434e-01, dot -0.078; [7afdbe76-bfde-4d2e-af29-681e534c7657 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1240.8776128208615, -1240.8775120857015, -1179.6014157332977, -245.10292799658677, -131.85364904175006, -33.031423692778894
Rejected: LBFGS Orientation magnitude: 4.875e+06, gradient 7.434e-01, dot -0.399; [7afdbe76-bfde-4d2e-af29-681e534c7657 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1240.8776128208615, -1240.8775120857015, -1179.6014157332977, -245.10292799658677, -131.85364904175006
Accepted: LBFGS Orientation magnitude: 1.255e+06, gradient 7.434e-01, dot 0.409; [7afdbe76-bfde-4d2e-af29-681e534c7657 = 1.000/1.000e+00]
Overwriting history with 4 points
th(0)=-1240.8776128208615;dx=-383162.23106208246
New Minimum: -1240.8776128208615 > -1240.8786705395496
END: th(4.005511000204042E-9)=-1240.8786705395496; dx=-282750.4084366601 evalInputDelta=0.001057718688116438
Fitness changed from -1240.8776128208615 to -1240.8786705395496
Iteration 14 complete. Error: -1240.8786705395496 Total: 0.4515; Orientation: 0.1080; Line Search: 0.2248
Adding measurement 1ed125b5 to history. Total: 4
Accepted: LBFGS Orientation magnitude: 2.114e+09, gradient 7.419e-01, dot 0.426; [7afdbe76-bfde-4d2e-af29-681e534c7657 = 1.000/1.000e+00]
th(0)=-1240.8786705395496;dx=-6.752893670383935E8
New Minimum: -1240.8786705395496 > -1245.9488352255883
END: th(8.629611850143898E-9)=-1245.9488352255883; dx=-5.39363391249686E8 evalInputDelta=5.070164686038652
Fitness changed from -1240.8786705395496 to -1245.9488352255883
Iteration 15 complete. Error: -1245.9488352255883 Total: 0.3923; Orientation: 0.0388; Line Search: 0.2431
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -1245.9488352255883 (> -Infinity) after 9.804s (< 5400.000s)
Returns
-1245.9488352255883