BasicOptimizer.scala:89 executed in 67.38 seconds (0.548 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 2159091555584
Reset training subject: 2159992363686
Adding measurement 13d33172 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.33847433178623904;dx=-7.514299295622646E-6
New Minimum: -0.33847433178623904 > -0.3384991276742466
WOLFE (weak): th(2.154434690031884)=-0.3384991276742466; dx=-7.3789385138946E-6 evalInputDelta=2.479588800757826E-5
New Minimum: -0.3384991276742466 > -0.3385241308316766
WOLFE (weak): th(4.308869380063768)=-0.3385241308316766; dx=-7.4038219941809314E-6 evalInputDelta=4.979904543755964E-5
New Minimum: -0.3385241308316766 > -0.33862268383223937
WOLFE (weak): th(12.926608140191302)=-0.33862268383223937; dx=-7.475846100504464E-6 evalInputDelta=1.4835204600033292E-4
New Minimum: -0.33862268383223937 > -0.3390640874025656
WOLFE (weak): th(51.70643256076521)=-0.3390640874025656; dx=-7.417662386472038E-6 evalInputDelta=5.897556163265705E-4
New Minimum: -0.3390640874025656 > -0.34140135632811475
WOLFE (weak): th(258.53216280382605)=-0.34140135632811475; dx=-7.385667961287741E-6 evalInputDelta=0.0029270245418757113
New Minimum: -0.34140135632811475 > -0.3552117728834036
WOLFE (weak): th(1551.1929768229563)=-0.3552117728834036; dx=-7.111259733826064E-6 evalInputDelta=0.016737441097164574
New Minimum: -0.3552117728834036 > -0.4492134093920818
WOLFE (weak): th(10858.350837760694)=-0.4492134093920818; dx=-1.0160991319873014E-5 evalInputDelta=0.11073907760584278
New Minimum: -0.4492134093920818 > -0.955406740910349
END: th(86866.80670208555)=-0.955406740910349; dx=-4.017398467868936E-6 evalInputDelta=0.61693240912411
Fitness changed from -0.33847433178623904 to -0.955406740910349
Iteration 1 complete. Error: -0.955406740910349 Total: 10.7594; Orientation: 0.0203; Line Search: 8.1308
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 6cf97617 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-0.955406740910349;dx=-2.3278430956759808E-4
New Minimum: -0.955406740910349 > -1.6457307115583737
END: th(187148.86177126726)=-1.6457307115583737; dx=-1.607846915988515E-7 evalInputDelta=0.6903239706480246
Fitness changed from -0.955406740910349 to -1.6457307115583737
Iteration 2 complete. Error: -1.6457307115583737 Total: 2.7146; Orientation: 0.0206; Line Search: 1.8093
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 1c4f9680 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-1.6457307115583737;dx=-6.606829149315721E-5
New Minimum: -1.6457307115583737 > -5.6752573232246
END: th(403200.00000000006)=-5.6752573232246; dx=-4.326492629585058E-6 evalInputDelta=4.029526611666227
Fitness changed from -1.6457307115583737 to -5.6752573232246
Iteration 3 complete. Error: -5.6752573232246 Total: 2.7107; Orientation: 0.0198; Line Search: 1.8072
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 49fa2e36 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.136e+04, gradient 2.287e-02, dot -0.022; [69bd94ee-bcfa-47e5-a7fc-c3359304cf2e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.6752573232246, -1.6457307115583737, -0.955406740910349, -0.33847433178623904
LBFGS Accumulation History: 3 points
Removed measurement 49fa2e36 to history. Total: 3
th(0)=-5.6752573232246;dx=-5.095585179159779E-4
New Minimum: -5.6752573232246 > -7.216410732649901
WOLF (strong): th(868668.0670208557)=-7.216410732649901; dx=9.251042179930555E-7 evalInputDelta=1.5411534094253012
New Minimum: -7.216410732649901 > -10.495132044469921
END: th(434334.03351042786)=-10.495132044469921; dx=-1.3451630094592331E-6 evalInputDelta=4.819874721245321
Fitness changed from -5.6752573232246 to -10.495132044469921
Iteration 4 complete. Error: -10.495132044469921 Total: 4.0987; Orientation: 0.4692; Line Search: 2.7584
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 4da59261 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.736e+03, gradient 2.212e-02, dot -0.014; [69bd94ee-bcfa-47e5-a7fc-c3359304cf2e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.495132044469921, -1.6457307115583737, -0.955406740910349, -0.33847433178623904
LBFGS Accumulation History: 3 points
Removed measurement 4da59261 to history. Total: 3
th(0)=-10.495132044469921;dx=-4.8143070712471585E-4
New Minimum: -10.495132044469921 > -11.617024628986062
WOLF (strong): th(935744.3088563365)=-11.617024628986062; dx=8.501282042986009E-6 evalInputDelta=1.1218925845161412
New Minimum: -11.617024628986062 > -25.714646448905096
WOLF (strong): th(467872.15442816826)=-25.714646448905096; dx=1.2207313918496746E-5 evalInputDelta=15.219514404435175
New Minimum: -25.714646448905096 > -39.60118720847031
END: th(155957.38480938942)=-39.60118720847031; dx=-3.441206320090501E-5 evalInputDelta=29.106055164000388
Fitness changed from -10.495132044469921 to -39.60118720847031
Iteration 5 complete. Error: -39.60118720847031 Total: 5.0347; Orientation: 0.5129; Line Search: 3.6532
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 3210be41 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.777e+04, gradient 2.565e-02, dot -0.021; [69bd94ee-bcfa-47e5-a7fc-c3359304cf2e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -39.60118720847031, -1.6457307115583737, -0.955406740910349, -0.33847433178623904
LBFGS Accumulation History: 3 points
Removed measurement 3210be41 to history. Total: 3
th(0)=-39.60118720847031;dx=-6.496515077422651E-4
Armijo: th(336000.0000000001)=-37.712938722278224; dx=1.685401111024996E-5 evalInputDelta=-1.888248486192083
New Minimum: -39.60118720847031 > -52.461144010204116
END: th(168000.00000000006)=-52.461144010204116; dx=-4.2354263999352267E-7 evalInputDelta=12.859956801733809
Fitness changed from -39.60118720847031 to -52.461144010204116
Iteration 6 complete. Error: -52.461144010204116 Total: 4.0698; Orientation: 0.4699; Line Search: 2.7234
Adding measurement 488765 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.667e+04, gradient 3.115e-02, dot -0.029; [69bd94ee-bcfa-47e5-a7fc-c3359304cf2e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -52.461144010204116, -1.6457307115583737, -0.955406740910349, -0.33847433178623904
LBFGS Accumulation History: 3 points
Removed measurement 488765 to history. Total: 3
th(0)=-52.461144010204116;dx=-9.598874363166515E-4
Armijo: th(361945.0279253566)=-48.98492324254602; dx=4.3671141295253264E-5 evalInputDelta=-3.4762207676580985
New Minimum: -52.461144010204116 > -77.37315097750317
WOLF (strong): th(180972.5139626783)=-77.37315097750317; dx=4.827266304489766E-5 evalInputDelta=24.912006967299057
New Minimum: -77.37315097750317 > -83.45238923800355
END: th(60324.17132089276)=-83.45238923800355; dx=-2.19538095097366E-4 evalInputDelta=30.991245227799432
Fitness changed from -52.461144010204116 to -83.45238923800355
Iteration 7 complete. Error: -83.45238923800355 Total: 4.9598; Orientation: 0.4693; Line Search: 3.6436
Adding measurement 128f0142 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 9.258e+04, gradient 3.030e-02, dot 0.052; [69bd94ee-bcfa-47e5-a7fc-c3359304cf2e = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-83.45238923800355;dx=-148.7398737250002
Armijo: th(129964.48734115784)=-10.482528683136213; dx=3.138377741606951E-5 evalInputDelta=-72.96986055486734
Armijo: th(64982.24367057892)=-10.445796752752022; dx=6.0997785030323454E-5 evalInputDelta=-73.00659248525153
Armijo: th(21660.747890192975)=-10.390713844779942; dx=1.7113526845451962E-4 evalInputDelta=-73.06167539322361
Armijo: th(5415.186972548244)=-10.332922855551063; dx=7.28287836297597E-4 evalInputDelta=-73.11946638245249
Armijo: th(1083.0373945096487)=-10.449032562593564; dx=0.003787631021370604 evalInputDelta=-73.00335667540999
Armijo: th(180.50623241827478)=-10.55977003945317; dx=0.02062830288807747 evalInputDelta=-72.89261919855038
Armijo: th(25.78660463118211)=-11.22929953152689; dx=0.18104298297889182 evalInputDelta=-72.22308970647666
Armijo: th(3.223325578897764)=-14.649618181584493; dx=2.12494767189824 evalInputDelta=-68.80277105641906
Armijo: th(0.358147286544196)=-52.67117277416794; dx=30.71410318697541 evalInputDelta=-30.781216463835605
Armijo: th(0.0358147286544196)=-82.68633372665794; dx=15.850535393225474 evalInputDelta=-0.7660555113456127
New Minimum: -83.45238923800355 > -83.72195192645117
END: th(0.0032558844231290545)=-83.72195192645117; dx=-47.907284249116245 evalInputDelta=0.2695626884476212
Fitness changed from -83.45238923800355 to -83.72195192645117
Iteration 8 complete. Error: -83.72195192645117 Total: 12.9152; Orientation: 0.4527; Line Search: 11.6146
Adding measurement 165bcff6 to history. Total: 4
Accepted: LBFGS Orientation magnitude: 8.627e+04, gradient 2.995e-02, dot 0.044; [69bd94ee-bcfa-47e5-a7fc-c3359304cf2e = 1.000/1.000e+00]
th(0)=-83.72195192645117;dx=-113.1560184431302
New Minimum: -83.72195192645117 > -83.81673530499444
END: th(0.007014590347923683)=-83.81673530499444; dx=-16.886342768175666 evalInputDelta=0.09478337854326924
Fitness changed from -83.72195192645117 to -83.81673530499444
Iteration 9 complete. Error: -83.81673530499444 Total: 3.2401; Orientation: 0.5735; Line Search: 1.8201
Adding measurement 3b36daf1 to history. Total: 5
Accepted: LBFGS Orientation magnitude: 8.135e+04, gradient 2.988e-02, dot 0.037; [69bd94ee-bcfa-47e5-a7fc-c3359304cf2e = 1.000/1.000e+00]
th(0)=-83.81673530499444;dx=-89.5660485880988
Armijo: th(0.015112476781929604)=-83.1627029157302; dx=-0.22295892398707862 evalInputDelta=-0.6540323892642448
Armijo: th(0.007556238390964802)=-83.571764047995; dx=-3.754410544502454 evalInputDelta=-0.24497125699943467
Armijo: th(0.002518746130321601)=-83.76897814357503; dx=-8.621963182322045 evalInputDelta=-0.047757161419411887
Armijo: th(6.296865325804002E-4)=-83.80787995282722; dx=-9.887273271093639 evalInputDelta=-0.008855352167216779
Armijo: th(1.2593730651608005E-4)=-83.81510412621046; dx=-15.575669124558479 evalInputDelta=-0.0016311787839811132
Armijo: th(2.0989551086013342E-5)=-83.81642654858891; dx=-12.026296807307606 evalInputDelta=-3.087564055306302E-4
Armijo: th(2.998507298001906E-6)=-83.81671200266388; dx=-11.721292049099857 evalInputDelta=-2.330233056113684E-5
Armijo: th(3.7481341225023825E-7)=-83.81673530499444; dx=-12.932583132819502 evalInputDelta=0.0
Armijo: th(4.164593469447092E-8)=-83.81673530499444; dx=-10.968602294077545 evalInputDelta=0.0
Armijo: th(4.164593469447092E-9)=-83.81673530499444; dx=-11.3201669760876 evalInputDelta=0.0
Armijo: th(3.78599406313372E-10)=-83.81673530499444; dx=-13.898157558147467 evalInputDelta=0.0
MIN ALPHA (3.1549950526114335E-11): th(0.0)=-83.81673530499444
Fitness changed from -83.81673530499444 to -83.81673530499444
Static Iteration Total: 13.3490; Orientation: 0.6883; Line Search: 11.8009
Iteration 10 failed. Error: -83.81673530499444
Previous Error: 0.0 -> -83.81673530499444
Retrying iteration 10
Reset training subject: 2222943617685
Adding measurement 3cd42cbe to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-83.81673530499444;dx=-8.837690919359488E-4
END: th(2.1544346900318838E-10)=-83.81673530499444; dx=-7.902976927739916E-4 evalInputDelta=0.0
Fitness changed from -83.81673530499444 to -83.81673530499444
Static Iteration Total: 3.5253; Orientation: 0.0196; Line Search: 1.7967
Iteration 11 failed. Error: -83.81673530499444
Previous Error: 0.0 -> -83.81673530499444
Optimization terminated 11
Final threshold in iteration 11: -83.81673530499444 (> -Infinity) after 67.378s (< 3600.000s)
Returns
-83.81673530499444