BasicOptimizer.scala:89 executed in 46.85 seconds (1.271 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 219020171863700
Reset training subject: 219020919757800
Adding measurement 45cd527a to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-172.22407733095858;dx=-0.12479275670660592
New Minimum: -172.22407733095858 > -172.43345606322208
WOLFE (weak): th(2.154434690031884)=-172.43345606322208; dx=-0.12477714832659818 evalInputDelta=0.20937873226350234
New Minimum: -172.43345606322208 > -172.64316287822638
WOLFE (weak): th(4.308869380063768)=-172.64316287822638; dx=-0.12488399900331604 evalInputDelta=0.4190855472677981
New Minimum: -172.64316287822638 > -173.4824550425056
WOLFE (weak): th(12.926608140191302)=-173.4824550425056; dx=-0.12488522706126343 evalInputDelta=1.2583777115470127
New Minimum: -173.4824550425056 > -177.2980147288813
WOLFE (weak): th(51.70643256076521)=-177.2980147288813; dx=-0.12625965616279908 evalInputDelta=5.07393739792272
New Minimum: -177.2980147288813 > -197.58636419820832
WOLFE (weak): th(258.53216280382605)=-197.58636419820832; dx=-0.12329040086812282 evalInputDelta=25.362286867249736
New Minimum: -197.58636419820832 > -283.1610094716523
END: th(1551.1929768229563)=-283.1610094716523; dx=-0.07419730145933917 evalInputDelta=110.9369321406937
Fitness changed from -172.22407733095858 to -283.1610094716523
Iteration 1 complete. Error: -283.1610094716523 Total: 7.8594; Orientation: 0.0371; Line Search: 5.1019
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 2cb532ec to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-283.1610094716523;dx=-0.1569489971580705
New Minimum: -283.1610094716523 > -430.9695632443052
END: th(3341.943960201201)=-430.9695632443052; dx=-0.041017437222804154 evalInputDelta=147.80855377265294
Fitness changed from -283.1610094716523 to -430.9695632443052
Iteration 2 complete. Error: -430.9695632443052 Total: 1.7716; Orientation: 0.0163; Line Search: 1.1756
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 599c2c58 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-430.9695632443052;dx=-0.13680508546227624
New Minimum: -430.9695632443052 > -579.9387328171204
END: th(7200.000000000001)=-579.9387328171204; dx=-0.020257923467437995 evalInputDelta=148.96916957281513
Fitness changed from -430.9695632443052 to -579.9387328171204
Iteration 3 complete. Error: -579.9387328171204 Total: 1.7574; Orientation: 0.0164; Line Search: 1.1830
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 42d45ace to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.516e+03, gradient 3.409e-01, dot -0.217; [d69617ec-2d24-4cf0-82e3-2fca69b90970 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -579.9387328171204, -430.9695632443052, -283.1610094716523, -172.22407733095858
LBFGS Accumulation History: 3 points
Removed measurement 42d45ace to history. Total: 3
th(0)=-579.9387328171204;dx=-0.11612562009487326
New Minimum: -579.9387328171204 > -712.7376165537912
END: th(15511.929768229566)=-712.7376165537912; dx=-0.009716138260717724 evalInputDelta=132.79888373667086
Fitness changed from -579.9387328171204 to -712.7376165537912
Iteration 4 complete. Error: -712.7376165537912 Total: 2.1115; Orientation: 0.3007; Line Search: 1.2049
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 12d15456 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.027e+03, gradient 3.198e-01, dot -0.191; [d69617ec-2d24-4cf0-82e3-2fca69b90970 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -712.7376165537912, -430.9695632443052, -283.1610094716523, -172.22407733095858
LBFGS Accumulation History: 3 points
Removed measurement 12d15456 to history. Total: 3
th(0)=-712.7376165537912;dx=-0.10212781212574268
New Minimum: -712.7376165537912 > -841.6039194139474
END: th(33419.43960201202)=-841.6039194139474; dx=-0.0024816459320439067 evalInputDelta=128.86630286015622
Fitness changed from -712.7376165537912 to -841.6039194139474
Iteration 5 complete. Error: -841.6039194139474 Total: 2.7860; Orientation: 0.2974; Line Search: 1.8949
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 34cf45ea to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.957e+03, gradient 2.925e-01, dot -0.132; [d69617ec-2d24-4cf0-82e3-2fca69b90970 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -841.6039194139474, -430.9695632443052, -283.1610094716523, -172.22407733095858
LBFGS Accumulation History: 3 points
Removed measurement 34cf45ea to history. Total: 3
th(0)=-841.6039194139474;dx=-0.08550928914713551
New Minimum: -841.6039194139474 > -1029.0157740354239
WOLF (strong): th(72000.00000000003)=-1029.0157740354239; dx=6.771839942641404E-4 evalInputDelta=187.41185462147644
New Minimum: -1029.0157740354239 > -1372.3113781893055
END: th(36000.000000000015)=-1372.3113781893055; dx=-0.0075828698399456615 evalInputDelta=530.7074587753581
Fitness changed from -841.6039194139474 to -1372.3113781893055
Iteration 6 complete. Error: -1372.3113781893055 Total: 2.7266; Orientation: 0.2873; Line Search: 1.8365
Adding measurement 12efe12c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.092e+04, gradient 2.686e-01, dot -0.143; [d69617ec-2d24-4cf0-82e3-2fca69b90970 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1372.3113781893055, -430.9695632443052, -283.1610094716523, -172.22407733095858
LBFGS Accumulation History: 3 points
Removed measurement 12efe12c to history. Total: 3
th(0)=-1372.3113781893055;dx=-0.07204886082535766
New Minimum: -1372.3113781893055 > -1420.8057176096502
WOLF (strong): th(77559.64884114785)=-1420.8057176096502; dx=6.741760508485605E-4 evalInputDelta=48.49433942034466
New Minimum: -1420.8057176096502 > -1879.007702406987
END: th(38779.824420573925)=-1879.007702406987; dx=-0.009811641892382442 evalInputDelta=506.6963242176814
Fitness changed from -1372.3113781893055 to -1879.007702406987
Iteration 7 complete. Error: -1879.007702406987 Total: 2.8473; Orientation: 0.3003; Line Search: 1.9916
Adding measurement 596c9847 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.062e+04, gradient 2.676e-01, dot -0.166; [d69617ec-2d24-4cf0-82e3-2fca69b90970 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1879.007702406987, -430.9695632443052, -283.1610094716523, -172.22407733095858
LBFGS Accumulation History: 3 points
Removed measurement 596c9847 to history. Total: 3
th(0)=-1879.007702406987;dx=-0.07156758083718018
Armijo: th(83548.59900503006)=-1714.144215171067; dx=0.002055908591979994 evalInputDelta=-164.86348723592005
New Minimum: -1879.007702406987 > -2323.13112319884
END: th(41774.29950251503)=-2323.13112319884; dx=-0.00963248629100323 evalInputDelta=444.1234207918533
Fitness changed from -1879.007702406987 to -2323.13112319884
Iteration 8 complete. Error: -2323.13112319884 Total: 2.9120; Orientation: 0.3735; Line Search: 1.9279
Adding measurement 2641ac1c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.663e+04, gradient 2.682e-01, dot -0.192; [d69617ec-2d24-4cf0-82e3-2fca69b90970 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2323.13112319884, -430.9695632443052, -283.1610094716523, -172.22407733095858
LBFGS Accumulation History: 3 points
Removed measurement 2641ac1c to history. Total: 3
th(0)=-2323.13112319884;dx=-0.07187949678748848
Armijo: th(90000.00000000004)=-2008.0620975179718; dx=0.0028699617863222632 evalInputDelta=-315.06902568086844
New Minimum: -2323.13112319884 > -2764.4306889677746
END: th(45000.00000000002)=-2764.4306889677746; dx=-0.008967216216645398 evalInputDelta=441.2995657689344
Fitness changed from -2323.13112319884 to -2764.4306889677746
Iteration 9 complete. Error: -2764.4306889677746 Total: 2.6321; Orientation: 0.2949; Line Search: 1.7958
Adding measurement 6362c932 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.076e+05, gradient 2.752e-01, dot -0.197; [d69617ec-2d24-4cf0-82e3-2fca69b90970 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2764.4306889677746, -430.9695632443052, -283.1610094716523, -172.22407733095858
LBFGS Accumulation History: 3 points
Removed measurement 6362c932 to history. Total: 3
th(0)=-2764.4306889677746;dx=-0.0756675596811476
Armijo: th(96949.56105143482)=-2220.724895763436; dx=0.003701453598408906 evalInputDelta=-543.7057932043385
New Minimum: -2764.4306889677746 > -3144.3518010711196
END: th(48474.78052571741)=-3144.3518010711196; dx=-0.006908433300600224 evalInputDelta=379.92111210334497
Fitness changed from -2764.4306889677746 to -3144.3518010711196
Iteration 10 complete. Error: -3144.3518010711196 Total: 2.9414; Orientation: 0.2798; Line Search: 1.7931
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 2e7231c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.569e+05, gradient 2.808e-01, dot -0.197; [d69617ec-2d24-4cf0-82e3-2fca69b90970 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3144.3518010711196, -430.9695632443052, -283.1610094716523, -172.22407733095858
LBFGS Accumulation History: 3 points
Removed measurement 2e7231c to history. Total: 3
th(0)=-3144.3518010711196;dx=-0.07882350365392407
Armijo: th(104435.74875628759)=-2380.4257798100416; dx=0.00497156653246138 evalInputDelta=-763.926021261078
New Minimum: -3144.3518010711196 > -3479.4469709189825
END: th(52217.874378143795)=-3479.4469709189825; dx=-0.005423753781938304 evalInputDelta=335.09516984786296
Fitness changed from -3144.3518010711196 to -3479.4469709189825
Iteration 11 complete. Error: -3479.4469709189825 Total: 2.7516; Orientation: 0.2950; Line Search: 1.8791
Adding measurement 75219ca to history. Total: 3
Accepted: LBFGS Orientation magnitude: 4.080e+04, gradient 2.877e-01, dot 0.250; [d69617ec-2d24-4cf0-82e3-2fca69b90970 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-3479.4469709189825;dx=-2932.01632360465
Armijo: th(112500.00000000007)=-2345.2604534246143; dx=0.010056756047077545 evalInputDelta=-1134.1865174943682
Armijo: th(56250.00000000004)=-2345.313456025419; dx=0.020082834217423868 evalInputDelta=-1134.1335148935636
Armijo: th(18750.00000000001)=-2345.629178870419; dx=0.06039299212870546 evalInputDelta=-1133.8177920485637
Armijo: th(4687.500000000003)=-2345.572429133726; dx=0.2405186018457987 evalInputDelta=-1133.8745417852565
Armijo: th(937.5000000000006)=-2347.1197290039463; dx=1.1974698174381833 evalInputDelta=-1132.3272419150362
Armijo: th(156.25000000000009)=-2362.764032583202; dx=7.070340970072818 evalInputDelta=-1116.6829383357804
Armijo: th(22.321428571428584)=-2440.054420654357; dx=43.49929241722499 evalInputDelta=-1039.3925502646257
Armijo: th(2.790178571428573)=-3003.1089723180767; dx=131.36818976511282 evalInputDelta=-476.33799860090585
New Minimum: -3479.4469709189825 > -3640.1190416065083
END: th(0.31001984126984145)=-3640.1190416065083; dx=-624.509710774685 evalInputDelta=160.67207068752577
Fitness changed from -3479.4469709189825 to -3640.1190416065083
Iteration 12 complete. Error: -3640.1190416065083 Total: 7.1984; Orientation: 0.2927; Line Search: 6.3364
Adding measurement 7ca6acf7 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.024e+06, gradient 2.692e-01, dot -0.199; [d69617ec-2d24-4cf0-82e3-2fca69b90970 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3640.1190416065083, -3479.4469709189825, -430.9695632443052, -283.1610094716523, -172.22407733095858
Rejected: LBFGS Orientation magnitude: 1.667e+06, gradient 2.692e-01, dot -0.199; [d69617ec-2d24-4cf0-82e3-2fca69b90970 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3640.1190416065083, -3479.4469709189825, -430.9695632443052, -283.1610094716523
LBFGS Accumulation History: 3 points
Removed measurement 7ca6acf7 to history. Total: 4
Removed measurement 75219ca to history. Total: 3
th(0)=-3640.1190416065083;dx=-0.07233168903965066
New Minimum: -3640.1190416065083 > -3640.1420817594544
END: th(0.6679175006299247)=-3640.1420817594544; dx=-0.0524787338107553 evalInputDelta=0.02304015294612327
Fitness changed from -3640.1190416065083 to -3640.1420817594544
Iteration 13 complete. Error: -3640.1420817594544 Total: 2.4358; Orientation: 0.6442; Line Search: 1.2292
Adding measurement 3e4c252 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.105e+05, gradient 2.696e-01, dot -0.168; [d69617ec-2d24-4cf0-82e3-2fca69b90970 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3640.1420817594544, -430.9695632443052, -283.1610094716523, -172.22407733095858
LBFGS Accumulation History: 3 points
Removed measurement 3e4c252 to history. Total: 3
th(0)=-3640.1420817594544;dx=-0.07239962415675819
New Minimum: -3640.1420817594544 > -3640.1917056564957
END: th(1.4389846334365024)=-3640.1917056564957; dx=-0.05251150953932667 evalInputDelta=0.049623897041328746
Fitness changed from -3640.1420817594544 to -3640.1917056564957
Iteration 14 complete. Error: -3640.1917056564957 Total: 2.0424; Orientation: 0.2916; Line Search: 1.2127
Adding measurement 579b9435 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.796e+05, gradient 2.693e-01, dot -0.168; [d69617ec-2d24-4cf0-82e3-2fca69b90970 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3640.1917056564957, -430.9695632443052, -283.1610094716523, -172.22407733095858
LBFGS Accumulation History: 3 points
Removed measurement 579b9435 to history. Total: 3
th(0)=-3640.1917056564957;dx=-0.07242053794303822
New Minimum: -3640.1917056564957 > -3640.2990108511844
END: th(3.1001984126984152)=-3640.2990108511844; dx=-0.05254341319366006 evalInputDelta=0.10730519468870625
Fitness changed from -3640.1917056564957 to -3640.2990108511844
Iteration 15 complete. Error: -3640.2990108511844 Total: 2.0523; Orientation: 0.2951; Line Search: 1.2138
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -3640.2990108511844 (> -Infinity) after 46.851s (< 5400.000s)
Returns
-3640.2990108511844