BasicOptimizer.scala:89 executed in 58.22 seconds (1.131 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 211997107937600
Reset training subject: 211997860586200
Adding measurement 1401d1d7 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=113.78535032272339;dx=-1.4760261215699254E-4
New Minimum: 113.78535032272339 > 113.78504037857056
WOLFE (weak): th(2.154434690031884)=113.78504037857056; dx=-1.4760040392284502E-4 evalInputDelta=3.0994415283203125E-4
New Minimum: 113.78504037857056 > 113.78471851348877
WOLFE (weak): th(4.308869380063768)=113.78471851348877; dx=-1.4760470938609906E-4 evalInputDelta=6.318092346191406E-4
New Minimum: 113.78471851348877 > 113.7834906578064
WOLFE (weak): th(12.926608140191302)=113.7834906578064; dx=-1.4760125581958337E-4 evalInputDelta=0.0018596649169921875
New Minimum: 113.7834906578064 > 113.77788782119751
WOLFE (weak): th(51.70643256076521)=113.77788782119751; dx=-1.4759664597513555E-4 evalInputDelta=0.007462501525878906
New Minimum: 113.77788782119751 > 113.7480616569519
WOLFE (weak): th(258.53216280382605)=113.7480616569519; dx=-1.475804261767314E-4 evalInputDelta=0.037288665771484375
New Minimum: 113.7480616569519 > 113.5618805885315
WOLFE (weak): th(1551.1929768229563)=113.5618805885315; dx=-1.4727139527168892E-4 evalInputDelta=0.22346973419189453
New Minimum: 113.5618805885315 > 112.23329305648804
WOLFE (weak): th(10858.350837760694)=112.23329305648804; dx=-1.4527676051984595E-4 evalInputDelta=1.5520572662353516
New Minimum: 112.23329305648804 > 102.42090225219727
END: th(86866.80670208555)=102.42090225219727; dx=-1.2470883181200164E-4 evalInputDelta=11.364448070526123
Fitness changed from 113.78535032272339 to 102.42090225219727
Iteration 1 complete. Error: 102.42090225219727 Total: 9.3740; Orientation: 0.0162; Line Search: 7.1248
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 5a871e43 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=102.42090225219727;dx=-1.2257161714534392E-4
New Minimum: 102.42090225219727 > 79.02520877702835
END: th(187148.86177126726)=79.02520877702835; dx=-1.0594578456891304E-4 evalInputDelta=23.395693475168912
Fitness changed from 102.42090225219727 to 79.02520877702835
Iteration 2 complete. Error: 79.02520877702835 Total: 2.3673; Orientation: 0.0170; Line Search: 1.6365
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 6528f1fa to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=79.02520877702835;dx=-0.030071848079640036
Armijo: th(403200.00000000006)=216.01421491729602; dx=4.5185314432042545E-4 evalInputDelta=-136.98900614026766
Armijo: th(201600.00000000003)=145.0047291729787; dx=5.342991940528122E-4 evalInputDelta=-65.97952039595035
Armijo: th(67200.00000000001)=84.73036955023048; dx=3.142085141416678E-4 evalInputDelta=-5.705160773202124
New Minimum: 79.02520877702835 > 47.15943120414962
END: th(16800.000000000004)=47.15943120414962; dx=-0.0012348045316314517 evalInputDelta=31.865777572878734
Fitness changed from 79.02520877702835 to 47.15943120414962
Iteration 3 complete. Error: 47.15943120414962 Total: 4.8082; Orientation: 0.0169; Line Search: 4.0616
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 13d1ca89 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.672e+07, gradient 2.825e-01, dot -0.040; [1f2257e2-b22e-4571-b62b-01174e9d6866 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 47.15943120414962, 79.02520877702835, 102.42090225219727, 113.78535032272339
LBFGS Accumulation History: 3 points
Removed measurement 13d1ca89 to history. Total: 3
th(0)=47.15943120414962;dx=-0.07923779543328537
Armijo: th(36194.502792535655)=104.95794225239769; dx=8.600983441597783E-4 evalInputDelta=-57.798511048248066
Armijo: th(18097.251396267828)=74.57616636429579; dx=-1.8749313993197644E-4 evalInputDelta=-27.416735160146168
New Minimum: 47.15943120414962 > 33.28833897981226
END: th(6032.417132089276)=33.28833897981226; dx=-0.004534306530313981 evalInputDelta=13.87109222433736
Fitness changed from 47.15943120414962 to 33.28833897981226
Iteration 4 complete. Error: 33.28833897981226 Total: 4.1342; Orientation: 0.3210; Line Search: 3.1025
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 247e309f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.373e+07, gradient 2.479e-01, dot -0.035; [1f2257e2-b22e-4571-b62b-01174e9d6866 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 33.28833897981226, 79.02520877702835, 102.42090225219727, 113.78535032272339
LBFGS Accumulation History: 3 points
Removed measurement 247e309f to history. Total: 3
th(0)=33.28833897981226;dx=-0.06145449183088205
New Minimum: 33.28833897981226 > 20.422893885452837
END: th(12996.448734115784)=20.422893885452837; dx=-0.00375592887526842 evalInputDelta=12.865445094359423
Fitness changed from 33.28833897981226 to 20.422893885452837
Iteration 5 complete. Error: 20.422893885452837 Total: 2.8887; Orientation: 0.3102; Line Search: 1.7518
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 595a3150 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.021e+07, gradient 2.573e-01, dot -0.037; [1f2257e2-b22e-4571-b62b-01174e9d6866 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 20.422893885452837, 79.02520877702835, 102.42090225219727, 113.78535032272339
LBFGS Accumulation History: 3 points
Removed measurement 595a3150 to history. Total: 3
th(0)=20.422893885452837;dx=-0.0659504984046623
Armijo: th(28000.000000000007)=50.71653168827193; dx=-6.069794287163326E-4 evalInputDelta=-30.29363780281909
New Minimum: 20.422893885452837 > 7.940397361947873
END: th(14000.000000000004)=7.940397361947873; dx=-0.0022933838815683673 evalInputDelta=12.482496523504963
Fitness changed from 20.422893885452837 to 7.940397361947873
Iteration 6 complete. Error: 7.940397361947873 Total: 3.3258; Orientation: 0.3016; Line Search: 2.3377
Adding measurement 15348b93 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.630e+07, gradient 2.083e-01, dot -0.037; [1f2257e2-b22e-4571-b62b-01174e9d6866 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 7.940397361947873, 79.02520877702835, 102.42090225219727, 113.78535032272339
LBFGS Accumulation History: 3 points
Removed measurement 15348b93 to history. Total: 3
th(0)=7.940397361947873;dx=-0.04318761094131493
New Minimum: 7.940397361947873 > -28.387741386344885
END: th(30162.08566044638)=-28.387741386344885; dx=-7.731668013131771E-4 evalInputDelta=36.32813874829276
Fitness changed from 7.940397361947873 to -28.387741386344885
Iteration 7 complete. Error: -28.387741386344885 Total: 2.8196; Orientation: 0.3242; Line Search: 1.6850
Adding measurement 6061284 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.241e+07, gradient 1.929e-01, dot -0.031; [1f2257e2-b22e-4571-b62b-01174e9d6866 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -28.387741386344885, 79.02520877702835, 102.42090225219727, 113.78535032272339
LBFGS Accumulation History: 3 points
Removed measurement 6061284 to history. Total: 3
th(0)=-28.387741386344885;dx=-0.03714263361586364
Armijo: th(64982.24367057892)=23.735155649701426; dx=-3.331461704732903E-6 evalInputDelta=-52.12289703604631
New Minimum: -28.387741386344885 > -72.60073902819644
END: th(32491.12183528946)=-72.60073902819644; dx=-0.001519772400920205 evalInputDelta=44.21299764185156
Fitness changed from -28.387741386344885 to -72.60073902819644
Iteration 8 complete. Error: -72.60073902819644 Total: 3.4858; Orientation: 0.4560; Line Search: 2.2514
Adding measurement 67520476 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.133e+06, gradient 1.795e-01, dot -0.022; [1f2257e2-b22e-4571-b62b-01174e9d6866 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -72.60073902819644, 79.02520877702835, 102.42090225219727, 113.78535032272339
LBFGS Accumulation History: 3 points
Removed measurement 67520476 to history. Total: 3
th(0)=-72.60073902819644;dx=-0.032176787208746276
Armijo: th(70000.00000000003)=23.433911991834833; dx=0.0010601191788038012 evalInputDelta=-96.03465102003128
New Minimum: -72.60073902819644 > -172.0959509484801
END: th(35000.000000000015)=-172.0959509484801; dx=-0.0023168220599445745 evalInputDelta=99.49521192028365
Fitness changed from -72.60073902819644 to -172.0959509484801
Iteration 9 complete. Error: -172.0959509484801 Total: 3.4906; Orientation: 0.3310; Line Search: 2.3816
Adding measurement 834e0d0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.298e+07, gradient 1.969e-01, dot -0.028; [1f2257e2-b22e-4571-b62b-01174e9d6866 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -172.0959509484801, 79.02520877702835, 102.42090225219727, 113.78535032272339
LBFGS Accumulation History: 3 points
Removed measurement 834e0d0 to history. Total: 3
th(0)=-172.0959509484801;dx=-0.03875357030343406
Armijo: th(75405.21415111597)=10.463846755871543; dx=0.0012511476646299992 evalInputDelta=-182.55979770435164
New Minimum: -172.0959509484801 > -229.52930213071386
END: th(37702.607075557986)=-229.52930213071386; dx=-0.0012049141078981276 evalInputDelta=57.433351182233764
Fitness changed from -172.0959509484801 to -229.52930213071386
Iteration 10 complete. Error: -229.52930213071386 Total: 3.3624; Orientation: 0.3068; Line Search: 2.3610
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 73de5b5b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.270e+07, gradient 2.130e-01, dot -0.026; [1f2257e2-b22e-4571-b62b-01174e9d6866 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -229.52930213071386, 79.02520877702835, 102.42090225219727, 113.78535032272339
LBFGS Accumulation History: 3 points
Removed measurement 73de5b5b to history. Total: 3
th(0)=-229.52930213071386;dx=-0.045330191079211074
Armijo: th(81227.80458822368)=43.995873258886775; dx=0.0016380278068163136 evalInputDelta=-273.52517538960063
New Minimum: -229.52930213071386 > -272.9876182374583
END: th(40613.90229411184)=-272.9876182374583; dx=-5.358771448317711E-4 evalInputDelta=43.45831610674446
Fitness changed from -229.52930213071386 to -272.9876182374583
Iteration 11 complete. Error: -272.9876182374583 Total: 3.2655; Orientation: 0.3367; Line Search: 2.2414
Adding measurement 4031a213 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.842e+07, gradient 2.156e-01, dot -0.013; [1f2257e2-b22e-4571-b62b-01174e9d6866 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -272.9876182374583, 79.02520877702835, 102.42090225219727, 113.78535032272339
LBFGS Accumulation History: 3 points
Removed measurement 4031a213 to history. Total: 3
th(0)=-272.9876182374583;dx=-0.046482055798845415
Armijo: th(87500.00000000006)=72.08373565170993; dx=0.003110622261146911 evalInputDelta=-345.0713538891682
New Minimum: -272.9876182374583 > -324.6491352569344
WOLF (strong): th(43750.00000000003)=-324.6491352569344; dx=5.019237984353706E-4 evalInputDelta=51.66151701947609
New Minimum: -324.6491352569344 > -498.8335697857592
END: th(14583.333333333343)=-498.8335697857592; dx=-0.01828897020753993 evalInputDelta=225.8459515483009
Fitness changed from -272.9876182374583 to -498.8335697857592
Iteration 12 complete. Error: -498.8335697857592 Total: 4.3110; Orientation: 0.3066; Line Search: 3.2218
Adding measurement a8424d1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.000e+08, gradient 2.209e-01, dot -0.015; [1f2257e2-b22e-4571-b62b-01174e9d6866 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -498.8335697857592, 79.02520877702835, 102.42090225219727, 113.78535032272339
LBFGS Accumulation History: 3 points
Removed measurement a8424d1 to history. Total: 3
th(0)=-498.8335697857592;dx=-0.04867510567717952
New Minimum: -498.8335697857592 > -554.7417491264835
END: th(31418.83922963166)=-554.7417491264835; dx=-0.00523596875400286 evalInputDelta=55.908179340724246
Fitness changed from -498.8335697857592 to -554.7417491264835
Iteration 13 complete. Error: -554.7417491264835 Total: 2.6735; Orientation: 0.2981; Line Search: 1.5866
Adding measurement 195b9fb7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.024e+09, gradient 2.365e-01, dot -0.019; [1f2257e2-b22e-4571-b62b-01174e9d6866 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -554.7417491264835, 79.02520877702835, 102.42090225219727, 113.78535032272339
LBFGS Accumulation History: 3 points
Removed measurement 195b9fb7 to history. Total: 3
th(0)=-554.7417491264835;dx=-0.05589555028256174
Armijo: th(67689.83715685307)=-159.90506449007142; dx=0.0032358480882562126 evalInputDelta=-394.83668463641203
New Minimum: -554.7417491264835 > -573.8321535192651
END: th(33844.918578426536)=-573.8321535192651; dx=-0.001573456153869043 evalInputDelta=19.090404392781693
Fitness changed from -554.7417491264835 to -573.8321535192651
Iteration 14 complete. Error: -573.8321535192651 Total: 3.7775; Orientation: 0.3071; Line Search: 2.6736
Adding measurement 168d1004 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.681e+08, gradient 2.537e-01, dot -0.013; [1f2257e2-b22e-4571-b62b-01174e9d6866 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -573.8321535192651, 79.02520877702835, 102.42090225219727, 113.78535032272339
LBFGS Accumulation History: 3 points
Removed measurement 168d1004 to history. Total: 3
th(0)=-573.8321535192651;dx=-0.06433957815409122
Armijo: th(72916.66666666672)=-39.82104698469726; dx=0.004012551935705843 evalInputDelta=-534.011106534568
Armijo: th(36458.33333333336)=-524.7276163863522; dx=0.0013733041151700763 evalInputDelta=-49.104537132912924
New Minimum: -573.8321535192651 > -790.4390323976311
END: th(12152.777777777786)=-790.4390323976311; dx=-0.024134043210637772 evalInputDelta=216.60687887836593
Fitness changed from -573.8321535192651 to -790.4390323976311
Iteration 15 complete. Error: -790.4390323976311 Total: 4.1170; Orientation: 0.3060; Line Search: 3.0402
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -790.4390323976311 (> -Infinity) after 58.223s (< 5400.000s)
Returns
-790.4390323976311