BasicOptimizer.scala:89 executed in 167.14 seconds (1.749 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 5292419536398
Reset training subject: 5293689388077
Adding measurement 34882d44 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.8764568593863072;dx=-2.304423634383805E-4
New Minimum: -0.8764568593863072 > -0.8764907888849423
WOLFE (weak): th(2.154434690031884)=-0.8764907888849423; dx=-2.3108637238752292E-4 evalInputDelta=3.392949863501027E-5
New Minimum: -0.8764907888849423 > -0.8765273478104457
WOLFE (weak): th(4.308869380063768)=-0.8765273478104457; dx=-2.290020809161715E-4 evalInputDelta=7.048842413848178E-5
New Minimum: -0.8765273478104457 > -0.8766643772938555
WOLFE (weak): th(12.926608140191302)=-0.8766643772938555; dx=-2.305494568490238E-4 evalInputDelta=2.0751790754824917E-4
New Minimum: -0.8766643772938555 > -0.8772833433589251
WOLFE (weak): th(51.70643256076521)=-0.8772833433589251; dx=-2.2868371049677267E-4 evalInputDelta=8.264839726178641E-4
New Minimum: -0.8772833433589251 > -0.8805988591143123
WOLFE (weak): th(258.53216280382605)=-0.8805988591143123; dx=-2.2991843504285972E-4 evalInputDelta=0.004141999728005108
New Minimum: -0.8805988591143123 > -0.9012667078383693
WOLFE (weak): th(1551.1929768229563)=-0.9012667078383693; dx=-2.2793309073638215E-4 evalInputDelta=0.024809848452062022
New Minimum: -0.9012667078383693 > -1.044944627734928
END: th(10858.350837760694)=-1.044944627734928; dx=-1.7993103951160767E-4 evalInputDelta=0.16848776834862067
Fitness changed from -0.8764568593863072 to -1.044944627734928
Iteration 1 complete. Error: -1.044944627734928 Total: 13.2298; Orientation: 0.0188; Line Search: 9.3844
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 50ef69d to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-1.044944627734928;dx=-2.47374355816758E-4
New Minimum: -1.044944627734928 > -1.481785126209218
END: th(23393.607721408407)=-1.481785126209218; dx=-1.4466872867769458E-4 evalInputDelta=0.4368404984742902
Fitness changed from -1.044944627734928 to -1.481785126209218
Iteration 2 complete. Error: -1.481785126209218 Total: 3.9657; Orientation: 0.0224; Line Search: 2.5607
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 73a525c6 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-1.481785126209218;dx=-3.2184599812600857E-4
New Minimum: -1.481785126209218 > -1.721956860957645
END: th(50400.00000000001)=-1.721956860957645; dx=-5.822804904765504E-5 evalInputDelta=0.2401717347484269
Fitness changed from -1.481785126209218 to -1.721956860957645
Iteration 3 complete. Error: -1.721956860957645 Total: 3.8043; Orientation: 0.0185; Line Search: 2.3899
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 5cc5ec4a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.500e+03, gradient 1.557e-02, dot -0.216; [f7561f8d-d6b2-4692-a0b6-2044911da57e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.721956860957645, -1.481785126209218, -1.044944627734928, -0.8764568593863072
LBFGS Accumulation History: 3 points
Removed measurement 5cc5ec4a to history. Total: 3
th(0)=-1.721956860957645;dx=-2.372246778539721E-4
New Minimum: -1.721956860957645 > -2.562625188783682
END: th(108583.50837760697)=-2.562625188783682; dx=-2.8243509082745418E-5 evalInputDelta=0.8406683278260372
Fitness changed from -1.721956860957645 to -2.562625188783682
Iteration 4 complete. Error: -2.562625188783682 Total: 4.3688; Orientation: 0.6314; Line Search: 2.3297
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 6ff0f19e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.838e+03, gradient 1.550e-02, dot -0.130; [f7561f8d-d6b2-4692-a0b6-2044911da57e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.562625188783682, -1.481785126209218, -1.044944627734928, -0.8764568593863072
LBFGS Accumulation History: 3 points
Removed measurement 6ff0f19e to history. Total: 3
th(0)=-2.562625188783682;dx=-2.3764971652833882E-4
Armijo: th(233936.07721408413)=-2.412616351000977; dx=-9.82117744559159E-6 evalInputDelta=-0.15000883778270513
New Minimum: -2.562625188783682 > -3.069528153999482
END: th(116968.03860704206)=-3.069528153999482; dx=-4.2517377169352905E-5 evalInputDelta=0.5069029652157999
Fitness changed from -2.562625188783682 to -3.069528153999482
Iteration 5 complete. Error: -3.069528153999482 Total: 5.5183; Orientation: 0.4805; Line Search: 3.6297
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 387fc3e6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.588e+03, gradient 1.519e-02, dot -0.129; [f7561f8d-d6b2-4692-a0b6-2044911da57e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.069528153999482, -1.481785126209218, -1.044944627734928, -0.8764568593863072
LBFGS Accumulation History: 3 points
Removed measurement 387fc3e6 to history. Total: 3
th(0)=-3.069528153999482;dx=-2.2414735380164234E-4
New Minimum: -3.069528153999482 > -3.320146244974547
END: th(252000.0000000001)=-3.320146244974547; dx=-8.985844291385338E-6 evalInputDelta=0.25061809097506504
Fitness changed from -3.069528153999482 to -3.320146244974547
Iteration 6 complete. Error: -3.320146244974547 Total: 4.2009; Orientation: 0.5404; Line Search: 2.2901
Adding measurement 11c3d400 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.543e+03, gradient 9.140e-03, dot -0.096; [f7561f8d-d6b2-4692-a0b6-2044911da57e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.320146244974547, -1.481785126209218, -1.044944627734928, -0.8764568593863072
LBFGS Accumulation History: 3 points
Removed measurement 11c3d400 to history. Total: 3
th(0)=-3.320146244974547;dx=-8.21504603159113E-5
New Minimum: -3.320146244974547 > -4.298361716611429
END: th(542917.5418880349)=-4.298361716611429; dx=-5.201578363546696E-6 evalInputDelta=0.9782154716368816
Fitness changed from -3.320146244974547 to -4.298361716611429
Iteration 7 complete. Error: -4.298361716611429 Total: 4.3046; Orientation: 0.4774; Line Search: 2.7421
Adding measurement 1839a424 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.025e+03, gradient 7.495e-03, dot -0.092; [f7561f8d-d6b2-4692-a0b6-2044911da57e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.298361716611429, -1.481785126209218, -1.044944627734928, -0.8764568593863072
LBFGS Accumulation History: 3 points
Removed measurement 1839a424 to history. Total: 3
th(0)=-4.298361716611429;dx=-5.519826896845343E-5
Armijo: th(1169680.3860704207)=-4.272208121220475; dx=-1.733476208380912E-6 evalInputDelta=-0.0261535953909533
New Minimum: -4.298361716611429 > -5.155072367615282
END: th(584840.1930352104)=-5.155072367615282; dx=-7.612756208159506E-6 evalInputDelta=0.8567106510038531
Fitness changed from -4.298361716611429 to -5.155072367615282
Iteration 8 complete. Error: -5.155072367615282 Total: 4.9900; Orientation: 0.4745; Line Search: 3.4242
Adding measurement 844f3b5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.711e+03, gradient 6.646e-03, dot -0.119; [f7561f8d-d6b2-4692-a0b6-2044911da57e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.155072367615282, -1.481785126209218, -1.044944627734928, -0.8764568593863072
LBFGS Accumulation History: 3 points
Removed measurement 844f3b5 to history. Total: 3
th(0)=-5.155072367615282;dx=-4.291625493147133E-5
New Minimum: -5.155072367615282 > -5.72957136427889
END: th(1260000.0000000005)=-5.72957136427889; dx=-2.3793748637777478E-6 evalInputDelta=0.5744989966636078
Fitness changed from -5.155072367615282 to -5.72957136427889
Iteration 9 complete. Error: -5.72957136427889 Total: 3.9471; Orientation: 0.5086; Line Search: 2.2628
Adding measurement 16fa4685 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.204e+03, gradient 4.692e-03, dot -0.094; [f7561f8d-d6b2-4692-a0b6-2044911da57e = 1.000/1.000e+00]
Orientat
...skipping 10551 bytes...
87402125106244
END: th(303819.44444444467)=-8.587402125106244; dx=-7.132638080538303E-6 evalInputDelta=0.02897892554019421
Fitness changed from -8.55842319956605 to -8.587402125106244
Iteration 21 complete. Error: -8.587402125106244 Total: 6.5637; Orientation: 0.4737; Line Search: 4.7248
Adding measurement b924f78 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.366e+04, gradient 3.084e-03, dot -0.275; [f7561f8d-d6b2-4692-a0b6-2044911da57e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.587402125106244, -1.481785126209218, -1.044944627734928, -0.8764568593863072
LBFGS Accumulation History: 3 points
Removed measurement b924f78 to history. Total: 3
th(0)=-8.587402125106244;dx=-9.345087566252585E-6
Armijo: th(654559.1506173263)=-8.520855048508583; dx=-5.888052079108325E-6 evalInputDelta=-0.06654707659766146
Armijo: th(327279.57530866313)=-8.572673611549963; dx=-6.896086208749042E-6 evalInputDelta=-0.014728513556281797
New Minimum: -8.587402125106244 > -8.590075133332611
END: th(109093.19176955438)=-8.590075133332611; dx=-7.648743274216882E-6 evalInputDelta=0.0026730082263668464
Fitness changed from -8.587402125106244 to -8.590075133332611
Iteration 22 complete. Error: -8.590075133332611 Total: 6.5386; Orientation: 0.4788; Line Search: 4.7463
Adding measurement 65e7e15c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.798e+04, gradient 3.083e-03, dot -0.280; [f7561f8d-d6b2-4692-a0b6-2044911da57e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.590075133332611, -1.481785126209218, -1.044944627734928, -0.8764568593863072
LBFGS Accumulation History: 3 points
Removed measurement 65e7e15c to history. Total: 3
th(0)=-8.590075133332611;dx=-9.3892757420152E-6
Armijo: th(235034.15679462874)=-8.587624150376064; dx=-7.176875968790225E-6 evalInputDelta=-0.002450982956547776
New Minimum: -8.590075133332611 > -8.592052000180002
END: th(117517.07839731437)=-8.592052000180002; dx=-7.62498552309419E-6 evalInputDelta=0.001976866847391179
Fitness changed from -8.590075133332611 to -8.592052000180002
Iteration 23 complete. Error: -8.592052000180002 Total: 5.0162; Orientation: 0.4913; Line Search: 3.4308
Adding measurement 3754e4b2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.866e+04, gradient 3.063e-03, dot -0.282; [f7561f8d-d6b2-4692-a0b6-2044911da57e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.592052000180002, -1.481785126209218, -1.044944627734928, -0.8764568593863072
LBFGS Accumulation History: 3 points
Removed measurement 3754e4b2 to history. Total: 3
th(0)=-8.592052000180002;dx=-9.222015894841735E-6
New Minimum: -8.592052000180002 > -8.595743092812027
END: th(253182.87037037057)=-8.595743092812027; dx=-7.029146607713763E-6 evalInputDelta=0.0036910926320246062
Fitness changed from -8.592052000180002 to -8.595743092812027
Iteration 24 complete. Error: -8.595743092812027 Total: 3.9790; Orientation: 0.5099; Line Search: 2.2898
Adding measurement 42e3a3a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.141e+04, gradient 3.028e-03, dot -0.287; [f7561f8d-d6b2-4692-a0b6-2044911da57e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.595743092812027, -1.481785126209218, -1.044944627734928, -0.8764568593863072
LBFGS Accumulation History: 3 points
Removed measurement 42e3a3a to history. Total: 3
th(0)=-8.595743092812027;dx=-9.020036452446417E-6
Armijo: th(545465.958847772)=-8.483566029687722; dx=-6.061613102780077E-6 evalInputDelta=-0.11217706312430487
Armijo: th(272732.979423886)=-8.567104043084782; dx=-6.848697268299386E-6 evalInputDelta=-0.028639049727244625
Armijo: th(90910.99314129533)=-8.594202895879803; dx=-7.447230349390451E-6 evalInputDelta=-0.0015401969322237363
Armijo: th(22727.748285323833)=-8.59532708527528; dx=-7.689670224642546E-6 evalInputDelta=-4.1600753674764235E-4
Armijo: th(4545.5496570647665)=-8.595668497116593; dx=-7.744831300857074E-6 evalInputDelta=-7.45956954339988E-5
Armijo: th(757.5916095107945)=-8.595724147934831; dx=-7.711923360223417E-6 evalInputDelta=-1.8944877195892218E-5
Armijo: th(108.22737278725636)=-8.595740724704662; dx=-7.711230525478384E-6 evalInputDelta=-2.3681073653136764E-6
New Minimum: -8.595743092812027 > -8.59574546091874
END: th(13.528421598407045)=-8.59574546091874; dx=-7.727740359445365E-6 evalInputDelta=2.3681067133907163E-6
Fitness changed from -8.595743092812027 to -8.59574546091874
Iteration 25 complete. Error: -8.59574546091874 Total: 12.2218; Orientation: 0.4695; Line Search: 10.6667
<a id="p-19"></a>Iteration 25
<a id="p-18"></a>
Adding measurement 66cea424 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.098e+04, gradient 3.025e-03, dot -0.287; [f7561f8d-d6b2-4692-a0b6-2044911da57e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.59574546091874, -1.481785126209218, -1.044944627734928, -0.8764568593863072
LBFGS Accumulation History: 3 points
Removed measurement 66cea424 to history. Total: 3
th(0)=-8.59574546091874;dx=-9.05858140593271E-6
Armijo: th(29.146100792984722)=-8.595743487496526; dx=-7.79878172080468E-6 evalInputDelta=-1.9734222149736524E-6
Armijo: th(14.573050396492361)=-8.595743092812027; dx=-7.713665746264586E-6 evalInputDelta=-2.3681067133907163E-6
Armijo: th(4.857683465497454)=-8.595743882181004; dx=-7.779561054709911E-6 evalInputDelta=-1.5787377360965138E-6
Armijo: th(1.2144208663743634)=-8.595743092812027; dx=-7.76893677340441E-6 evalInputDelta=-2.3681067133907163E-6
New Minimum: -8.59574546091874 > -8.59574585560313
END: th(0.2428841732748727)=-8.59574585560313; dx=-7.776596932247198E-6 evalInputDelta=3.946843900592967E-7
Fitness changed from -8.59574546091874 to -8.59574585560313
Iteration 26 complete. Error: -8.59574585560313 Total: 9.0896; Orientation: 0.4859; Line Search: 7.1121
Adding measurement 7e90495d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.071e+04, gradient 3.029e-03, dot -0.286; [f7561f8d-d6b2-4692-a0b6-2044911da57e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.59574585560313, -1.481785126209218, -1.044944627734928, -0.8764568593863072
LBFGS Accumulation History: 3 points
Removed measurement 7e90495d to history. Total: 3
th(0)=-8.59574585560313;dx=-9.050251766899144E-6
Armijo: th(0.5232780885631007)=-8.595744276865467; dx=-7.75752881308668E-6 evalInputDelta=-1.5787376632658834E-6
Armijo: th(0.2616390442815503)=-8.59574546091874; dx=-7.699319378074558E-6 evalInputDelta=-3.946843900592967E-7
Armijo: th(0.08721301476051678)=-8.59574546091874; dx=-7.728021555011043E-6 evalInputDelta=-3.946843900592967E-7
Armijo: th(0.021803253690129195)=-8.59574546091874; dx=-7.788980300328982E-6 evalInputDelta=-3.946843900592967E-7
Armijo: th(0.004360650738025839)=-8.595745066234334; dx=-7.706113171175817E-6 evalInputDelta=-7.893687961058049E-7
Armijo: th(7.267751230043065E-4)=-8.595745066234334; dx=-7.76836615380404E-6 evalInputDelta=-7.893687961058049E-7
Armijo: th(1.0382501757204378E-4)=-8.59574546091874; dx=-7.764938212483832E-6 evalInputDelta=-3.946843900592967E-7
END: th(1.2978127196505473E-5)=-8.59574585560313; dx=-7.760311389445953E-6 evalInputDelta=0.0
Fitness changed from -8.59574585560313 to -8.59574585560313
Static Iteration Total: 12.2254; Orientation: 0.4777; Line Search: 10.6316
Iteration 27 failed. Error: -8.59574585560313
Previous Error: 0.0 -> -8.59574585560313
Retrying iteration 27
Reset training subject: 5453860907991
Adding measurement 39964392 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-8.59574585560313;dx=-9.029479594055802E-6
Armijo: th(2.796052744379763E-5)=-8.59574546091874; dx=-7.736385631886075E-6 evalInputDelta=-3.946843900592967E-7
END: th(1.3980263721898814E-5)=-8.59574585560313; dx=-7.7512860344365E-6 evalInputDelta=0.0
Fitness changed from -8.59574585560313 to -8.59574585560313
Static Iteration Total: 5.6715; Orientation: 0.0182; Line Search: 3.4940
Iteration 28 failed. Error: -8.59574585560313
Previous Error: 0.0 -> -8.59574585560313
Optimization terminated 28
Final threshold in iteration 28: -8.59574585560313 (> -Infinity) after 167.113s (< 3600.000s)
Returns
-8.59574585560313