BasicOptimizer.scala:89 executed in 100.42 seconds (1.665 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 2721639407627
Reset training subject: 2722498059493
Adding measurement 2b94ddf6 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.4364453442377171;dx=-7.890969994528939E-5
New Minimum: -0.4364453442377171 > -0.43661037252621554
WOLFE (weak): th(2.154434690031884)=-0.43661037252621554; dx=-7.954957208550383E-5 evalInputDelta=1.6502828849845885E-4
New Minimum: -0.43661037252621554 > -0.43677494783424986
WOLFE (weak): th(4.308869380063768)=-0.43677494783424986; dx=-7.825609406312547E-5 evalInputDelta=3.2960359653277926E-4
New Minimum: -0.43677494783424986 > -0.4374346250858253
WOLFE (weak): th(12.926608140191302)=-0.4374346250858253; dx=-7.923416204562077E-5 evalInputDelta=9.892808481082316E-4
New Minimum: -0.4374346250858253 > -0.4404018442612887
WOLFE (weak): th(51.70643256076521)=-0.4404018442612887; dx=-7.91257851069043E-5 evalInputDelta=0.0039565000235716186
New Minimum: -0.4404018442612887 > -0.45639601763238113
WOLFE (weak): th(258.53216280382605)=-0.45639601763238113; dx=-8.081198364409055E-5 evalInputDelta=0.019950673394664054
New Minimum: -0.45639601763238113 > -0.56210895494829
WOLFE (weak): th(1551.1929768229563)=-0.56210895494829; dx=-8.729083955869238E-5 evalInputDelta=0.12566361071057297
New Minimum: -0.56210895494829 > -1.4499799312187989
WOLFE (weak): th(10858.350837760694)=-1.4499799312187989; dx=-8.86030379415984E-5 evalInputDelta=1.013534586981082
New Minimum: -1.4499799312187989 > -4.928486935153702
END: th(86866.80670208555)=-4.928486935153702; dx=-2.3605419527613063E-5 evalInputDelta=4.492041590915985
Fitness changed from -0.4364453442377171 to -4.928486935153702
Iteration 1 complete. Error: -4.928486935153702 Total: 10.8940; Orientation: 0.0196; Line Search: 8.2536
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 4253e0e1 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-4.928486935153702;dx=-2.546511858782501E-4
New Minimum: -4.928486935153702 > -13.002464825596547
END: th(187148.86177126726)=-13.002464825596547; dx=-1.0081649061286689E-5 evalInputDelta=8.073977890442844
Fitness changed from -4.928486935153702 to -13.002464825596547
Iteration 2 complete. Error: -13.002464825596547 Total: 2.7875; Orientation: 0.0200; Line Search: 1.8745
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 7cec63c4 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-13.002464825596547;dx=-4.7373535551925756E-4
New Minimum: -13.002464825596547 > -17.96615628615698
WOLF (strong): th(403200.00000000006)=-17.96615628615698; dx=7.561219975793687E-6 evalInputDelta=4.963691460560433
New Minimum: -17.96615628615698 > -24.072576810849263
WOLF (strong): th(201600.00000000003)=-24.072576810849263; dx=4.7510755257419875E-6 evalInputDelta=11.070111985252716
New Minimum: -24.072576810849263 > -25.97892374090111
END: th(67200.00000000001)=-25.97892374090111; dx=-7.708798651371646E-5 evalInputDelta=12.976458915304562
Fitness changed from -13.002464825596547 to -25.97892374090111
Iteration 3 complete. Error: -25.97892374090111 Total: 4.6219; Orientation: 0.0191; Line Search: 3.7171
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 2312f0e7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.803e+03, gradient 2.255e-02, dot -0.169; [5ca64b38-e81e-4498-b9d6-73891c993b97 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -25.97892374090111, -13.002464825596547, -4.928486935153702, -0.4364453442377171
LBFGS Accumulation History: 3 points
Removed measurement 2312f0e7 to history. Total: 3
th(0)=-25.97892374090111;dx=-5.065430978866381E-4
New Minimum: -25.97892374090111 > -37.059603314614364
END: th(144778.01117014262)=-37.059603314614364; dx=-8.512264020906758E-6 evalInputDelta=11.080679573713255
Fitness changed from -25.97892374090111 to -37.059603314614364
Iteration 4 complete. Error: -37.059603314614364 Total: 3.2101; Orientation: 0.4846; Line Search: 1.8457
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 5a6a7f54 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.208e+04, gradient 2.783e-02, dot -0.148; [5ca64b38-e81e-4498-b9d6-73891c993b97 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -37.059603314614364, -13.002464825596547, -4.928486935153702, -0.4364453442377171
LBFGS Accumulation History: 3 points
Removed measurement 5a6a7f54 to history. Total: 3
th(0)=-37.059603314614364;dx=-7.719164402976641E-4
Armijo: th(311914.76961877884)=-35.08502959987865; dx=3.350947872588361E-5 evalInputDelta=-1.9745737147357119
New Minimum: -37.059603314614364 > -49.583118758275226
WOLF (strong): th(155957.38480938942)=-49.583118758275226; dx=2.7299595632329406E-5 evalInputDelta=12.523515443660862
New Minimum: -49.583118758275226 > -53.85038641268279
END: th(51985.79493646314)=-53.85038641268279; dx=-1.4653338214900072E-4 evalInputDelta=16.790783098068424
Fitness changed from -37.059603314614364 to -53.85038641268279
Iteration 5 complete. Error: -53.85038641268279 Total: 5.0428; Orientation: 0.4737; Line Search: 3.6741
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 68d82ca2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.021e+04, gradient 2.357e-02, dot -0.184; [5ca64b38-e81e-4498-b9d6-73891c993b97 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -53.85038641268279, -13.002464825596547, -4.928486935153702, -0.4364453442377171
LBFGS Accumulation History: 3 points
Removed measurement 68d82ca2 to history. Total: 3
th(0)=-53.85038641268279;dx=-5.511740146504576E-4
New Minimum: -53.85038641268279 > -65.09193266975123
END: th(112000.00000000004)=-65.09193266975123; dx=-2.357961409377254E-5 evalInputDelta=11.241546257068443
Fitness changed from -53.85038641268279 to -65.09193266975123
Iteration 6 complete. Error: -65.09193266975123 Total: 3.1525; Orientation: 0.4764; Line Search: 1.7975
Adding measurement 161c2996 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.496e+04, gradient 3.104e-02, dot -0.159; [5ca64b38-e81e-4498-b9d6-73891c993b97 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -65.09193266975123, -13.002464825596547, -4.928486935153702, -0.4364453442377171
LBFGS Accumulation History: 3 points
Removed measurement 161c2996 to history. Total: 3
th(0)=-65.09193266975123;dx=-9.60681547055686E-4
Armijo: th(241296.68528357107)=-56.1441694964355; dx=5.79332316297919E-5 evalInputDelta=-8.947763173315728
New Minimum: -65.09193266975123 > -74.79801904011653
WOLF (strong): th(120648.34264178554)=-74.79801904011653; dx=4.465441372631315E-5 evalInputDelta=9.706086370365298
New Minimum: -74.79801904011653 > -80.2343703398529
END: th(40216.114213928515)=-80.2343703398529; dx=-2.046609561598462E-4 evalInputDelta=15.142437670101671
Fitness changed from -65.09193266975123 to -80.2343703398529
Iteration 7 complete. Error: -80.2343703398529 Total: 4.9835; Orientation: 0.4697; Line Search: 3.6696
Adding measurement 9356c4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.507e+04, gradient 2.414e-02, dot -0.214; [5ca64b38-e81e-4498-b9d6-73891c993b97 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -80.2343703398529, -13.002464825596547, -4.928486935153702, -0.4364453442377171
LBFGS Accumulation History: 3 points
Removed measurement 9356c4 to history. Total: 3
th(0)=-80.2343703398529;dx=-5.799723017958009E-4
New Minimum: -80.2343703398529 > -91.11289161537236
END: th(86642.99156077192)=-91.11289161537236; dx=-6.894270035658856E-5 evalInputDelta=10.87852127551946
Fitness changed from -80.2343703398529 to -91.11289161537236
Iteration 8 complete. Error: -91.11289161537236 Total: 3.1460; Orientation: 0.4754; Line Search: 1.8241
Adding measurement 1834bf4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.689e+05, gradient 2.942e-02, dot -0.
...skipping 3470 bytes...
w Minimum: -126.9226596316476 > -127.12934993087711
END: th(25925.92592592594)=-127.12934993087711; dx=-3.235648748042122E-4 evalInputDelta=8.797301533622388
Fitness changed from -118.33204839725472 to -127.12934993087711
Iteration 12 complete. Error: -127.12934993087711 Total: 5.0204; Orientation: 0.4675; Line Search: 3.6989
Adding measurement 16598066 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.270e+04, gradient 2.247e-02, dot -0.106; [5ca64b38-e81e-4498-b9d6-73891c993b97 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -127.12934993087711, -13.002464825596547, -4.928486935153702, -0.4364453442377171
LBFGS Accumulation History: 3 points
Removed measurement 16598066 to history. Total: 3
th(0)=-127.12934993087711;dx=-5.031141850788407E-4
New Minimum: -127.12934993087711 > -134.76452814830773
END: th(55855.714186011835)=-134.76452814830773; dx=-1.7200750352652203E-4 evalInputDelta=7.635178217430621
Fitness changed from -127.12934993087711 to -134.76452814830773
Iteration 13 complete. Error: -134.76452814830773 Total: 3.2135; Orientation: 0.5145; Line Search: 1.8306
Adding measurement c3b9adb to history. Total: 3
Accepted: LBFGS Orientation magnitude: 6.099e+03, gradient 2.406e-02, dot 0.360; [5ca64b38-e81e-4498-b9d6-73891c993b97 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-134.76452814830773;dx=-52.735754401696866
Armijo: th(120337.48827884991)=-16.98086725441621; dx=4.732322722487912E-5 evalInputDelta=-117.78366089389152
Armijo: th(60168.744139424954)=-16.98432944527421; dx=9.444110534882064E-5 evalInputDelta=-117.78019870303352
Armijo: th(20056.248046474986)=-16.987233184571767; dx=2.8614670818021427E-4 evalInputDelta=-117.77729496373597
Armijo: th(5014.0620116187465)=-17.046739924710987; dx=0.0011240698223993983 evalInputDelta=-117.71778822359674
Armijo: th(1002.8124023237493)=-17.328104709933918; dx=0.005730873665053829 evalInputDelta=-117.43642343837382
Armijo: th(167.13540038729155)=-18.11061278872312; dx=0.03372279599487469 evalInputDelta=-116.65391535958462
Armijo: th(23.87648576961308)=-24.999331390197142; dx=0.3666962800406869 evalInputDelta=-109.76519675811059
Armijo: th(2.984560721201635)=-64.49467097732278; dx=4.4264841610954235 evalInputDelta=-70.26985717098495
Armijo: th(0.3316178579112928)=-131.58200888447126; dx=-3.195244348692468 evalInputDelta=-3.1825192638364683
New Minimum: -134.76452814830773 > -135.29051886673741
END: th(0.03316178579112928)=-135.29051886673741; dx=-40.06469022825428 evalInputDelta=0.5259907184296821
Fitness changed from -134.76452814830773 to -135.29051886673741
Iteration 14 complete. Error: -135.29051886673741 Total: 11.4556; Orientation: 0.4721; Line Search: 10.1261
Adding measurement 11ca622b to history. Total: 4
Rejected: LBFGS Orientation magnitude: 4.705e+05, gradient 2.362e-02, dot -0.236; [5ca64b38-e81e-4498-b9d6-73891c993b97 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -135.29051886673741, -134.76452814830773, -13.002464825596547, -4.928486935153702, -0.4364453442377171
Rejected: LBFGS Orientation magnitude: 9.466e+05, gradient 2.362e-02, dot -0.307; [5ca64b38-e81e-4498-b9d6-73891c993b97 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -135.29051886673741, -134.76452814830773, -13.002464825596547, -4.928486935153702
LBFGS Accumulation History: 3 points
Removed measurement 11ca622b to history. Total: 4
Removed measurement c3b9adb to history. Total: 3
th(0)=-135.29051886673741;dx=-5.556667837042714E-4
New Minimum: -135.29051886673741 > -135.29054429692994
END: th(0.07144490169181533)=-135.29054429692994; dx=-4.804270625692243E-4 evalInputDelta=2.543019252243539E-5
Fitness changed from -135.29051886673741 to -135.29054429692994
Iteration 15 complete. Error: -135.29054429692994 Total: 3.7661; Orientation: 1.0915; Line Search: 1.8331
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Adding measurement 560cbb2a to history. Total: 3
Accepted: LBFGS Orientation magnitude: 6.337e+03, gradient 2.363e-02, dot 0.465; [5ca64b38-e81e-4498-b9d6-73891c993b97 = 1.000/1.000e+00]
th(0)=-135.29054429692994;dx=-69.36131755012453
New Minimum: -135.29054429692994 > -138.44853263853543
END: th(0.15392337463076458)=-138.44853263853543; dx=-37.48320906005203 evalInputDelta=3.1579883416054884
Fitness changed from -135.29054429692994 to -138.44853263853543
Iteration 16 complete. Error: -138.44853263853543 Total: 3.4581; Orientation: 0.5758; Line Search: 1.8575
Adding measurement 61b97a72 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 2.520e+05, gradient 2.341e-02, dot -0.262; [5ca64b38-e81e-4498-b9d6-73891c993b97 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -138.44853263853543, -135.29054429692994, -13.002464825596547, -4.928486935153702, -0.4364453442377171
Rejected: LBFGS Orientation magnitude: 3.437e+05, gradient 2.341e-02, dot -0.290; [5ca64b38-e81e-4498-b9d6-73891c993b97 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -138.44853263853543, -135.29054429692994, -13.002464825596547, -4.928486935153702
LBFGS Accumulation History: 3 points
Removed measurement 61b97a72 to history. Total: 4
Removed measurement 560cbb2a to history. Total: 3
th(0)=-138.44853263853543;dx=-5.457937128780494E-4
New Minimum: -138.44853263853543 > -138.44863203902648
END: th(0.33161785791129283)=-138.44863203902648; dx=-4.519244486160986E-4 evalInputDelta=9.940049105239268E-5
Fitness changed from -138.44853263853543 to -138.44863203902648
Iteration 17 complete. Error: -138.44863203902648 Total: 3.7742; Orientation: 1.0819; Line Search: 1.8369
Adding measurement 2c151b6b to history. Total: 3
Accepted: LBFGS Orientation magnitude: 9.905e+03, gradient 2.342e-02, dot 0.199; [5ca64b38-e81e-4498-b9d6-73891c993b97 = 1.000/1.000e+00]
th(0)=-138.44863203902648;dx=-45.90652770159315
Armijo: th(0.7144490169181534)=-100.7539145408713; dx=8.022811294398263 evalInputDelta=-37.69471749815517
Armijo: th(0.3572245084590767)=-118.60726528123156; dx=8.702360796549552 evalInputDelta=-19.841366757794916
Armijo: th(0.11907483615302557)=-134.21522115190237; dx=-8.177015883379358 evalInputDelta=-4.2334108871241085
Armijo: th(0.029768709038256392)=-137.82535498561322; dx=-26.165352816556446 evalInputDelta=-0.6232770534132612
Armijo: th(0.0059537418076512786)=-138.34819493570413; dx=-31.97937062125052 evalInputDelta=-0.10043710332234923
Armijo: th(9.922903012752132E-4)=-138.4326959863717; dx=-33.46418232503825 evalInputDelta=-0.015936052654780042
Armijo: th(1.4175575732503046E-4)=-138.44638308545805; dx=-33.625377330700225 evalInputDelta=-0.0022489535684258044
Armijo: th(1.7719469665628808E-5)=-138.4483524749635; dx=-33.615090723938906 evalInputDelta=-2.7956406296425484E-4
Armijo: th(1.968829962847645E-6)=-138.4486009763807; dx=-33.484850068513424 evalInputDelta=-3.1062645774682096E-5
Armijo: th(1.968829962847645E-7)=-138.44862582649787; dx=-33.613068110524246 evalInputDelta=-6.212528603555256E-6
Armijo: th(1.7898454207705864E-8)=-138.44863203902648; dx=-33.647799040670684 evalInputDelta=0.0
Armijo: th(1.4915378506421553E-9)=-138.44863203902648; dx=-33.607631010555394 evalInputDelta=0.0
END: th(1.1473368081862734E-10)=-138.44863203902648; dx=-33.560994083331146 evalInputDelta=0.0
Fitness changed from -138.44863203902648 to -138.44863203902648
Static Iteration Total: 15.1190; Orientation: 0.4629; Line Search: 13.8132
Iteration 18 failed. Error: -138.44863203902648
Previous Error: 0.0 -> -138.44863203902648
Retrying iteration 18
Reset training subject: 2818483226693
Adding measurement 8cd65a1 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-138.44863203902648;dx=-5.458003142205462E-4
END: th(2.471862220706965E-10)=-138.44863203902648; dx=-4.5203849424071014E-4 evalInputDelta=0.0
Fitness changed from -138.44863203902648 to -138.44863203902648
Static Iteration Total: 3.5711; Orientation: 0.0222; Line Search: 1.8306
Iteration 19 failed. Error: -138.44863203902648
Previous Error: 0.0 -> -138.44863203902648
Optimization terminated 19
Final threshold in iteration 19: -138.44863203902648 (> -Infinity) after 100.415s (< 3600.000s)
Returns
-138.44863203902648