BasicOptimizer.scala:89 executed in 189.66 seconds (2.027 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 10630839005066
Reset training subject: 10632012006111
Adding measurement 2a64dde7 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.1974358438003935;dx=-1.6058780187725789E-4
New Minimum: -0.1974358438003935 > -0.19776082909793627
WOLFE (weak): th(2.154434690031884)=-0.19776082909793627; dx=-1.599111941046152E-4 evalInputDelta=3.2498529754276806E-4
New Minimum: -0.19776082909793627 > -0.19808472642206748
WOLFE (weak): th(4.308869380063768)=-0.19808472642206748; dx=-1.602132975130934E-4 evalInputDelta=6.488826216739796E-4
New Minimum: -0.19808472642206748 > -0.19938731074816562
WOLFE (weak): th(12.926608140191302)=-0.19938731074816562; dx=-1.599466234768918E-4 evalInputDelta=0.0019514669477721236
New Minimum: -0.19938731074816562 > -0.2052924081183004
WOLFE (weak): th(51.70643256076521)=-0.2052924081183004; dx=-1.6224524967233529E-4 evalInputDelta=0.007856564317906906
New Minimum: -0.2052924081183004 > -0.23788448037666518
WOLFE (weak): th(258.53216280382605)=-0.23788448037666518; dx=-1.7026415666122807E-4 evalInputDelta=0.04044863657627168
New Minimum: -0.23788448037666518 > -0.4509480238946797
WOLFE (weak): th(1551.1929768229563)=-0.4509480238946797; dx=-1.6925212982944597E-4 evalInputDelta=0.2535121800942862
New Minimum: -0.4509480238946797 > -1.0245978496919876
END: th(10858.350837760694)=-1.0245978496919876; dx=-5.772169730038323E-6 evalInputDelta=0.8271620058915942
Fitness changed from -0.1974358438003935 to -1.0245978496919876
Iteration 1 complete. Error: -1.0245978496919876 Total: 13.5266; Orientation: 0.0173; Line Search: 10.0397
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 67fe2ad1 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-1.0245978496919876;dx=-3.2543326330989876E-4
New Minimum: -1.0245978496919876 > -1.6429879020993132
WOLF (strong): th(23393.607721408407)=-1.6429879020993132; dx=3.7217590944066577E-6 evalInputDelta=0.6183900524073256
New Minimum: -1.6429879020993132 > -2.054835209051724
WOLF (strong): th(11696.803860704204)=-2.054835209051724; dx=1.2892192462709223E-5 evalInputDelta=1.0302373593597365
New Minimum: -2.054835209051724 > -2.153417080804684
END: th(3898.9346202347347)=-2.153417080804684; dx=-2.3173927456266352E-4 evalInputDelta=1.1288192311126963
Fitness changed from -1.0245978496919876 to -2.153417080804684
Iteration 2 complete. Error: -2.153417080804684 Total: 6.0891; Orientation: 0.0188; Line Search: 4.7677
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 26d93b65 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-2.153417080804684;dx=-5.794974716898155E-4
New Minimum: -2.153417080804684 > -2.6589104288084835
END: th(8400.000000000002)=-2.6589104288084835; dx=-4.4755942344922405E-5 evalInputDelta=0.5054933480037995
Fitness changed from -2.153417080804684 to -2.6589104288084835
Iteration 3 complete. Error: -2.6589104288084835 Total: 3.6720; Orientation: 0.0176; Line Search: 2.3588
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 7c24e282 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.675e+03, gradient 2.954e-02, dot -0.270; [809471a2-9e36-4a41-bebc-c6d5d649f0d6 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.6589104288084835, -2.153417080804684, -1.0245978496919876, -0.1974358438003935
LBFGS Accumulation History: 3 points
Removed measurement 7c24e282 to history. Total: 3
th(0)=-2.6589104288084835;dx=-8.618195129018932E-4
New Minimum: -2.6589104288084835 > -4.699441982176996
END: th(18097.251396267828)=-4.699441982176996; dx=-1.6372759001796816E-5 evalInputDelta=2.0405315533685124
Fitness changed from -2.6589104288084835 to -4.699441982176996
Iteration 4 complete. Error: -4.699441982176996 Total: 4.1183; Orientation: 0.4648; Line Search: 2.3567
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 117108f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.230e+03, gradient 3.699e-02, dot -0.117; [809471a2-9e36-4a41-bebc-c6d5d649f0d6 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.699441982176996, -2.153417080804684, -1.0245978496919876, -0.1974358438003935
LBFGS Accumulation History: 3 points
Removed measurement 117108f to history. Total: 3
th(0)=-4.699441982176996;dx=-0.0013441952072846437
New Minimum: -4.699441982176996 > -6.938511130389788
END: th(38989.346202347355)=-6.938511130389788; dx=-4.749973800977362E-6 evalInputDelta=2.239069148212792
Fitness changed from -4.699441982176996 to -6.938511130389788
Iteration 5 complete. Error: -6.938511130389788 Total: 4.2326; Orientation: 0.5699; Line Search: 2.3903
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 370026bf to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.365e+03, gradient 3.924e-02, dot -0.094; [809471a2-9e36-4a41-bebc-c6d5d649f0d6 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.938511130389788, -2.153417080804684, -1.0245978496919876, -0.1974358438003935
LBFGS Accumulation History: 3 points
Removed measurement 370026bf to history. Total: 3
th(0)=-6.938511130389788;dx=-0.0015096401217923334
New Minimum: -6.938511130389788 > -14.098102176724057
WOLF (strong): th(84000.00000000003)=-14.098102176724057; dx=3.53501696657657E-7 evalInputDelta=7.159591046334269
New Minimum: -14.098102176724057 > -14.683127673668508
END: th(42000.000000000015)=-14.683127673668508; dx=-5.044342201112103E-5 evalInputDelta=7.74461654327872
Fitness changed from -6.938511130389788 to -14.683127673668508
Iteration 6 complete. Error: -14.683127673668508 Total: 5.3186; Orientation: 0.4597; Line Search: 3.5771
Adding measurement 2d0e46f0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.781e+04, gradient 4.142e-02, dot -0.080; [809471a2-9e36-4a41-bebc-c6d5d649f0d6 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -14.683127673668508, -2.153417080804684, -1.0245978496919876, -0.1974358438003935
LBFGS Accumulation History: 3 points
Removed measurement 2d0e46f0 to history. Total: 3
th(0)=-14.683127673668508;dx=-0.001698607316099728
New Minimum: -14.683127673668508 > -24.131302558784384
WOLF (strong): th(90486.25698133915)=-24.131302558784384; dx=4.3523674407029345E-6 evalInputDelta=9.448174885115876
New Minimum: -24.131302558784384 > -29.1533065569435
END: th(45243.12849066957)=-29.1533065569435; dx=-7.90722656986045E-5 evalInputDelta=14.470178883274993
Fitness changed from -14.683127673668508 to -29.1533065569435
Iteration 7 complete. Error: -29.1533065569435 Total: 5.3607; Orientation: 0.5931; Line Search: 3.6299
Adding measurement 5884d8ac to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.392e+05, gradient 4.993e-02, dot -0.062; [809471a2-9e36-4a41-bebc-c6d5d649f0d6 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -29.1533065569435, -2.153417080804684, -1.0245978496919876, -0.1974358438003935
LBFGS Accumulation History: 3 points
Removed measurement 5884d8ac to history. Total: 3
th(0)=-29.1533065569435;dx=-0.002392909379362234
New Minimum: -29.1533065569435 > -31.541827814118015
WOLF (strong): th(97473.3655058684)=-31.541827814118015; dx=3.3809928872491213E-6 evalInputDelta=2.3885212571745136
New Minimum: -31.541827814118015 > -42.0555415980054
WOLF (strong): th(48736.6827529342)=-42.0555415980054; dx=4.5885699593688216E-5 evalInputDelta=12.902235041061896
New Minimum: -42.0555415980054 > -47.058243395651466
END: th(16245.560917644732)=-47.058243395651466; dx=-5.237299685067333E-4 evalInputDelta=17.904936838707965
Fitness changed from -29.1533065569435 to -47.058243395651466
Iteration 8 complete. Error: -47.058243395651466 Total: 6.8528; Orientation: 0.4873; Line Search: 5.2128
Adding measurement 2fe8e723 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 2.603e+03, gradient 4.873e-02, dot 0.210; [809471a2-9e3
...skipping 15114 bytes...
5)=-97.79417260405909; dx=-4.799244404248221E-4 evalInputDelta=0.15560420477078196
Fitness changed from -97.6385683992883 to -97.79417260405909
Iteration 25 complete. Error: -97.79417260405909 Total: 3.9803; Orientation: 0.4757; Line Search: 2.3707
<a id="p-19"></a>Iteration 25
<a id="p-18"></a>
Adding measurement 7c8e344a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.922e+05, gradient 2.500e-02, dot -0.039; [809471a2-9e36-4a41-bebc-c6d5d649f0d6 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -97.79417260405909, -2.153417080804684, -1.0245978496919876, -0.1974358438003935
LBFGS Accumulation History: 3 points
Removed measurement 7c8e344a to history. Total: 3
th(0)=-97.79417260405909;dx=-6.211255548828946E-4
New Minimum: -97.79417260405909 > -98.12781644047217
END: th(639.548725971779)=-98.12781644047217; dx=-4.988214759098179E-4 evalInputDelta=0.3336438364130885
Fitness changed from -97.79417260405909 to -98.12781644047217
Iteration 26 complete. Error: -98.12781644047217 Total: 4.2027; Orientation: 0.4978; Line Search: 2.4105
Adding measurement 26b22ba2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.189e+05, gradient 2.495e-02, dot -0.039; [809471a2-9e36-4a41-bebc-c6d5d649f0d6 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -98.12781644047217, -2.153417080804684, -1.0245978496919876, -0.1974358438003935
LBFGS Accumulation History: 3 points
Removed measurement 26b22ba2 to history. Total: 3
th(0)=-98.12781644047217;dx=-6.187292899654699E-4
New Minimum: -98.12781644047217 > -98.84243102977081
END: th(1377.865961199296)=-98.84243102977081; dx=-4.96694108778869E-4 evalInputDelta=0.714614589298634
Fitness changed from -98.12781644047217 to -98.84243102977081
Iteration 27 complete. Error: -98.84243102977081 Total: 4.0393; Orientation: 0.4616; Line Search: 2.4208
Adding measurement 63b5155c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.357e+05, gradient 2.512e-02, dot -0.039; [809471a2-9e36-4a41-bebc-c6d5d649f0d6 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -98.84243102977081, -2.153417080804684, -1.0245978496919876, -0.1974358438003935
LBFGS Accumulation History: 3 points
Removed measurement 63b5155c to history. Total: 3
th(0)=-98.84243102977081;dx=-6.22807755349859E-4
New Minimum: -98.84243102977081 > -100.38339103314104
END: th(2968.522225021889)=-100.38339103314104; dx=-4.989718339651048E-4 evalInputDelta=1.5409600033702304
Fitness changed from -98.84243102977081 to -100.38339103314104
Iteration 28 complete. Error: -100.38339103314104 Total: 4.0910; Orientation: 0.5499; Line Search: 2.4049
Adding measurement 3eaca34a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.756e+05, gradient 2.509e-02, dot -0.040; [809471a2-9e36-4a41-bebc-c6d5d649f0d6 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -100.38339103314104, -2.153417080804684, -1.0245978496919876, -0.1974358438003935
LBFGS Accumulation History: 3 points
Removed measurement 3eaca34a to history. Total: 3
th(0)=-100.38339103314104;dx=-6.253211448854811E-4
New Minimum: -100.38339103314104 > -103.64489687600398
END: th(6395.487259717792)=-103.64489687600398; dx=-5.001107340050008E-4 evalInputDelta=3.2615058428629453
Fitness changed from -100.38339103314104 to -103.64489687600398
Iteration 29 complete. Error: -103.64489687600398 Total: 4.4193; Orientation: 0.4798; Line Search: 2.4129
Adding measurement 6c57cc09 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.221e+06, gradient 2.558e-02, dot -0.042; [809471a2-9e36-4a41-bebc-c6d5d649f0d6 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -103.64489687600398, -2.153417080804684, -1.0245978496919876, -0.1974358438003935
LBFGS Accumulation History: 3 points
Removed measurement 6c57cc09 to history. Total: 3
th(0)=-103.64489687600398;dx=-6.429406215451355E-4
New Minimum: -103.64489687600398 > -110.79890109732587
END: th(13778.659611992964)=-110.79890109732587; dx=-4.998573740781134E-4 evalInputDelta=7.154004221321884
Fitness changed from -103.64489687600398 to -110.79890109732587
Iteration 30 complete. Error: -110.79890109732587 Total: 4.0812; Orientation: 0.5318; Line Search: 2.4090
<a id="p-21"></a>Iteration 30
<a id="p-20"></a>
Adding measurement 6d89b4bb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.932e+05, gradient 2.545e-02, dot -0.043; [809471a2-9e36-4a41-bebc-c6d5d649f0d6 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -110.79890109732587, -2.153417080804684, -1.0245978496919876, -0.1974358438003935
LBFGS Accumulation History: 3 points
Removed measurement 6d89b4bb to history. Total: 3
th(0)=-110.79890109732587;dx=-6.416057280089429E-4
New Minimum: -110.79890109732587 > -125.64420328973597
END: th(29685.222250218896)=-125.64420328973597; dx=-4.587267888918438E-4 evalInputDelta=14.845302192410102
Fitness changed from -110.79890109732587 to -125.64420328973597
Iteration 31 complete. Error: -125.64420328973597 Total: 4.1506; Orientation: 0.4574; Line Search: 2.4051
Adding measurement 32450f46 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 8.174e+04, gradient 2.541e-02, dot 0.058; [809471a2-9e36-4a41-bebc-c6d5d649f0d6 = 1.000/1.000e+00]
th(0)=-125.64420328973597;dx=-122.18026611710451
Armijo: th(63954.87259717793)=-72.93110980469; dx=1.4961861086713828E-4 evalInputDelta=-52.713093485045974
Armijo: th(31977.436298588964)=-72.9565032549361; dx=2.8329544289796383E-4 evalInputDelta=-52.687700034799875
Armijo: th(10659.145432862988)=-73.04218400712871; dx=8.785908988074101E-4 evalInputDelta=-52.60201928260726
Armijo: th(2664.786358215747)=-73.06724816158228; dx=0.0031956237097226837 evalInputDelta=-52.576955128153685
Armijo: th(532.9572716431494)=-72.74928854892671; dx=0.015010654778214737 evalInputDelta=-52.894914740809256
Armijo: th(88.8262119405249)=-73.78172318251485; dx=0.09210375716509649 evalInputDelta=-51.86248010722112
Armijo: th(12.689458848646414)=-79.14817492368348; dx=0.409748541886139 evalInputDelta=-46.496028366052485
Armijo: th(1.5861823560808017)=-83.9505905884899; dx=4.219140858070643 evalInputDelta=-41.69361270124607
Armijo: th(0.17624248400897796)=-115.21196074382858; dx=21.386437722484505 evalInputDelta=-10.432242545907386
Armijo: th(0.017624248400897796)=-123.74026950380785; dx=69.77479447458481 evalInputDelta=-1.9039337859281176
Armijo: th(0.0016022044000816179)=-125.45143093419661; dx=90.64035394370889 evalInputDelta=-0.19277235553936123
Armijo: th(1.3351703334013481E-4)=-125.62841646473738; dx=79.4901432714851 evalInputDelta=-0.015786824998585303
Armijo: th(1.0270541026164216E-5)=-125.64298689640421; dx=82.7154903677684 evalInputDelta=-0.001216393331759491
Armijo: th(7.336100732974439E-7)=-125.64412945149884; dx=83.15815721816197 evalInputDelta=-7.383823712814319E-5
Armijo: th(4.89073382198296E-8)=-125.644195517292; dx=82.41718087433226 evalInputDelta=-7.772443964881859E-6
Armijo: th(3.05670863873935E-9)=-125.64420328973597; dx=82.57631639711887 evalInputDelta=0.0
Armijo: th(1.798063905140794E-10)=-125.64420328973597; dx=83.81212745696762 evalInputDelta=0.0
MIN ALPHA (9.989243917448855E-12): th(0.0)=-125.64420328973597
Fitness changed from -125.64420328973597 to -125.64420328973597
Static Iteration Total: 25.1131; Orientation: 0.4563; Line Search: 23.4997
Iteration 32 failed. Error: -125.64420328973597
Previous Error: 0.0 -> -125.64420328973597
Retrying iteration 32
Reset training subject: 10815795467257
Adding measurement 66a35e23 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-125.64420328973597;dx=-6.319467750594375E-4
END: th(2.1544346900318838E-10)=-125.64420328973597; dx=-4.97648809474164E-4 evalInputDelta=0.0
Fitness changed from -125.64420328973597 to -125.64420328973597
Static Iteration Total: 4.7056; Orientation: 0.0197; Line Search: 2.3946
Iteration 33 failed. Error: -125.64420328973597
Previous Error: 0.0 -> -125.64420328973597
Optimization terminated 33
Final threshold in iteration 33: -125.64420328973597 (> -Infinity) after 189.662s (< 3600.000s)
Returns
-125.64420328973597