BasicOptimizer.scala:89 executed in 8.15 seconds (0.000 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 158704576027400
Reset training subject: 158704708474900
Adding measurement 26486a72 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-2.404648226291845;dx=-8.242698752853614E-4
New Minimum: -2.404648226291845 > -2.4076368080150345
WOLFE (weak): th(2.154434690031884)=-2.4076368080150345; dx=-8.212718129907462E-4 evalInputDelta=0.0029885817231893874
New Minimum: -2.4076368080150345 > -2.410630276209896
WOLFE (weak): th(4.308869380063768)=-2.410630276209896; dx=-8.332598669509704E-4 evalInputDelta=0.005982049918050958
New Minimum: -2.410630276209896 > -2.422661734214399
WOLFE (weak): th(12.926608140191302)=-2.422661734214399; dx=-8.253933846926574E-4 evalInputDelta=0.01801350792255363
New Minimum: -2.422661734214399 > -2.4769590950254607
WOLFE (weak): th(51.70643256076521)=-2.4769590950254607; dx=-8.211874014831752E-4 evalInputDelta=0.07231086873361559
New Minimum: -2.4769590950254607 > -2.7720825669848206
WOLFE (weak): th(258.53216280382605)=-2.7720825669848206; dx=-8.486582974251981E-4 evalInputDelta=0.3674343406929754
New Minimum: -2.7720825669848206 > -4.517519505613512
END: th(1551.1929768229563)=-4.517519505613512; dx=-7.299035456330067E-4 evalInputDelta=2.112871279321667
Fitness changed from -2.404648226291845 to -4.517519505613512
Iteration 1 complete. Error: -4.517519505613512 Total: 1.1364; Orientation: 0.0015; Line Search: 0.7877
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 153102ba to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-4.517519505613512;dx=-8.79505145310055E-4
New Minimum: -4.517519505613512 > -7.108732655829049
END: th(3341.943960201201)=-7.108732655829049; dx=-2.9187346771880276E-4 evalInputDelta=2.5912131502155367
Fitness changed from -4.517519505613512 to -7.108732655829049
Iteration 2 complete. Error: -7.108732655829049 Total: 0.3473; Orientation: 0.0015; Line Search: 0.2268
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 36a197ee to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-7.108732655829049;dx=-4.567857882790703E-4
New Minimum: -7.108732655829049 > -9.216117605044914
END: th(7200.000000000001)=-9.216117605044914; dx=-1.1656712492099633E-4 evalInputDelta=2.1073849492158656
Fitness changed from -7.108732655829049 to -9.216117605044914
Iteration 3 complete. Error: -9.216117605044914 Total: 0.3459; Orientation: 0.0015; Line Search: 0.2239
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 43e0378c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.556e+02, gradient 1.987e-02, dot -0.322; [ba8c9e13-fdc0-4ece-8d28-68159c231c41 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -9.216117605044914, -7.108732655829049, -4.517519505613512, -2.404648226291845
LBFGS Accumulation History: 3 points
Removed measurement 43e0378c to history. Total: 3
th(0)=-9.216117605044914;dx=-3.7677035823301206E-4
New Minimum: -9.216117605044914 > -11.066279655131599
END: th(15511.929768229566)=-11.066279655131599; dx=-2.944033304175043E-5 evalInputDelta=1.8501620500866842
Fitness changed from -9.216117605044914 to -11.066279655131599
Iteration 4 complete. Error: -11.066279655131599 Total: 0.3701; Orientation: 0.0278; Line Search: 0.2229
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement d4c6597 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.749e+02, gradient 2.356e-02, dot -0.228; [ba8c9e13-fdc0-4ece-8d28-68159c231c41 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.066279655131599, -7.108732655829049, -4.517519505613512, -2.404648226291845
LBFGS Accumulation History: 3 points
Removed measurement d4c6597 to history. Total: 3
th(0)=-11.066279655131599;dx=-5.295396492701696E-4
Armijo: th(33419.43960201202)=-9.888942809485641; dx=2.1757609392391297E-5 evalInputDelta=-1.1773368456459572
New Minimum: -11.066279655131599 > -12.132680956521796
WOLF (strong): th(16709.71980100601)=-12.132680956521796; dx=3.5961965877684796E-5 evalInputDelta=1.0664013013901972
New Minimum: -12.132680956521796 > -12.795519667489799
END: th(5569.906600335336)=-12.795519667489799; dx=-1.0639542428229815E-4 evalInputDelta=1.7292400123582006
Fitness changed from -11.066279655131599 to -12.795519667489799
Iteration 5 complete. Error: -12.795519667489799 Total: 0.6012; Orientation: 0.0289; Line Search: 0.4522
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 5b7d4ef3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.376e+02, gradient 1.600e-02, dot -0.249; [ba8c9e13-fdc0-4ece-8d28-68159c231c41 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.795519667489799, -7.108732655829049, -4.517519505613512, -2.404648226291845
LBFGS Accumulation History: 3 points
Removed measurement 5b7d4ef3 to history. Total: 3
th(0)=-12.795519667489799;dx=-2.3566125230679906E-4
New Minimum: -12.795519667489799 > -14.276135186108016
END: th(12000.000000000004)=-14.276135186108016; dx=-2.4430421195468153E-5 evalInputDelta=1.480615518618217
Fitness changed from -12.795519667489799 to -14.276135186108016
Iteration 6 complete. Error: -14.276135186108016 Total: 0.3741; Orientation: 0.0272; Line Search: 0.2264
Adding measurement 2700bbd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.250e+02, gradient 1.957e-02, dot -0.304; [ba8c9e13-fdc0-4ece-8d28-68159c231c41 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -14.276135186108016, -7.108732655829049, -4.517519505613512, -2.404648226291845
LBFGS Accumulation History: 3 points
Removed measurement 2700bbd to history. Total: 3
th(0)=-14.276135186108016;dx=-3.661053162090907E-4
New Minimum: -14.276135186108016 > -14.44198687360599
WOLF (strong): th(25853.216280382614)=-14.44198687360599; dx=1.255075397253521E-5 evalInputDelta=0.1658516874979732
New Minimum: -14.44198687360599 > -15.623124699353449
END: th(12926.608140191307)=-15.623124699353449; dx=-2.741122123444981E-5 evalInputDelta=1.3469895132454326
Fitness changed from -14.276135186108016 to -15.623124699353449
Iteration 7 complete. Error: -15.623124699353449 Total: 0.4719; Orientation: 0.0276; Line Search: 0.3359
Adding measurement 7ffd991 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.073e+02, gradient 1.751e-02, dot -0.285; [ba8c9e13-fdc0-4ece-8d28-68159c231c41 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -15.623124699353449, -7.108732655829049, -4.517519505613512, -2.404648226291845
LBFGS Accumulation History: 3 points
Removed measurement 7ffd991 to history. Total: 3
th(0)=-15.623124699353449;dx=-2.857835473354814E-4
New Minimum: -15.623124699353449 > -15.745059770889572
WOLF (strong): th(27849.533001676686)=-15.745059770889572; dx=5.9487982072967065E-5 evalInputDelta=0.12193507153612337
New Minimum: -15.745059770889572 > -17.494945569627237
END: th(13924.766500838343)=-17.494945569627237; dx=-1.9944787465448125E-5 evalInputDelta=1.8718208702737886
Fitness changed from -15.623124699353449 to -17.494945569627237
Iteration 8 complete. Error: -17.494945569627237 Total: 0.4713; Orientation: 0.0286; Line Search: 0.3339
Adding measurement 5ed4b0f1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.006e+03, gradient 1.948e-02, dot -0.336; [ba8c9e13-fdc0-4ece-8d28-68159c231c41 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.494945569627237, -7.108732655829049, -4.517519505613512, -2.404648226291845
LBFGS Accumulation History: 3 points
Removed measurement 5ed4b0f1 to history. Total: 3
th(0)=-17.494945569627237;dx=-3.574484648142616E-4
New Minimum: -17.494945569627237 > -17.707245690348334
WOLF (strong): th(30000.000000000015)=-17.707245690348334; dx=5.556961959844612E-5 evalInputDelta=0.21230012072109616
New Minimum: -17.707245690348334 > -19.618431100428086
END: th(15000.000000000007)=-19.618431100428086; dx=-4.413061847724499E-7 evalInputDelta=2.123485530800849
Fitness changed from -17.494945569627237 to -19.618431100428086
Iteration 9 complete. Error: -19.618431100428086 Total: 0.4756; Orientation: 0.0280; Line Search: 0.3380
Adding measurement 41fb762 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.958e+03, gradient 1.954e-02, dot -0.324; [ba8c9e13-fdc0-4ece-8d28-68159c231c41 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.618431100428086, -7.108732655829049, -4.517519505613512, -2.404648226291845
LBFGS Accumulation History: 3 points
Removed measurement 41fb762 to history. Total: 3
th(0)=-19.618431100428086;dx=-3.5868463893576737E-4
Armijo: th(32316.520350478273)=-18.989953991448512; dx=5.8150995378866696E-5 evalInputDelta=-0.628477108979574
New Minimum: -19.618431100428086 > -21.808446255036422
WOLF (strong): th(16158.260175239137)=-21.808446255036422; dx=4.9208402987976685E-6 evalInputDelta=2.1900151546083357
END: th(5386.086725079712)=-21.51098199543179; dx=-2.2040303705380803E-4 evalInputDelta=1.8925508950037049
Fitness changed from -19.618431100428086 to -21.808446255036422
Iteration 10 complete. Error: -21.808446255036422 Total: 0.5895; Orientation: 0.0284; Line Search: 0.4518
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 46e52b2f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.494e+03, gradient 2.153e-02, dot -0.299; [ba8c9e13-fdc0-4ece-8d28-68159c231c41 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -21.808446255036422, -7.108732655829049, -4.517519505613512, -2.404648226291845
LBFGS Accumulation History: 3 points
Removed measurement 46e52b2f to history. Total: 3
th(0)=-21.808446255036422;dx=-4.407804460267365E-4
New Minimum: -21.808446255036422 > -25.659005581282944
END: th(11603.972084031953)=-25.659005581282944; dx=-7.09594856362996E-5 evalInputDelta=3.850559326246522
Fitness changed from -21.808446255036422 to -25.659005581282944
Iteration 11 complete. Error: -25.659005581282944 Total: 0.3734; Orientation: 0.0295; Line Search: 0.2247
Adding measurement 38d4725e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.786e+04, gradient 2.057e-02, dot -0.359; [ba8c9e13-fdc0-4ece-8d28-68159c231c41 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -25.659005581282944, -7.108732655829049, -4.517519505613512, -2.404648226291845
LBFGS Accumulation History: 3 points
Removed measurement 38d4725e to history. Total: 3
th(0)=-25.659005581282944;dx=-4.0347137845179386E-4
New Minimum: -25.659005581282944 > -27.109585613809603
END: th(25000.000000000015)=-27.109585613809603; dx=-1.829774904518576E-5 evalInputDelta=1.4505800325266591
Fitness changed from -25.659005581282944 to -27.109585613809603
Iteration 12 complete. Error: -27.109585613809603 Total: 0.3729; Orientation: 0.0276; Line Search: 0.2266
Adding measurement 5221ccb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.405e+04, gradient 2.434e-02, dot -0.304; [ba8c9e13-fdc0-4ece-8d28-68159c231c41 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -27.109585613809603, -7.108732655829049, -4.517519505613512, -2.404648226291845
LBFGS Accumulation History: 3 points
Removed measurement 5221ccb to history. Total: 3
th(0)=-27.109585613809603;dx=-5.686615131981548E-4
Armijo: th(53860.86725079713)=-25.63304123999245; dx=7.661269184113326E-5 evalInputDelta=-1.476544373817152
New Minimum: -27.109585613809603 > -32.53931620789591
WOLF (strong): th(26930.433625398564)=-32.53931620789591; dx=7.737046597301927E-5 evalInputDelta=5.429730594086305
New Minimum: -32.53931620789591 > -33.04210402391961
END: th(8976.811208466188)=-33.04210402391961; dx=-2.976517595943293E-4 evalInputDelta=5.93251841011001
Fitness changed from -27.109585613809603 to -33.04210402391961
Iteration 13 complete. Error: -33.04210402391961 Total: 0.6183; Orientation: 0.0266; Line Search: 0.4830
Adding measurement 14d8cf88 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.153e+04, gradient 2.078e-02, dot 0.408; [ba8c9e13-fdc0-4ece-8d28-68159c231c41 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-33.04210402391961;dx=-100.05713767834806
New Minimum: -33.04210402391961 > -34.69778048209516
Armijo: th(19339.95347338659)=-34.69778048209516; dx=6.008625469043986E-4 evalInputDelta=1.6556764581755488
New Minimum: -34.69778048209516 > -34.703387302321396
WOLF (strong): th(9669.976736693296)=-34.703387302321396; dx=0.0011904669088965447 evalInputDelta=1.6612832784017826
New Minimum: -34.703387302321396 > -34.70996464158423
WOLF (strong): th(3223.3255788977654)=-34.70996464158423; dx=0.003605766171800374 evalInputDelta=1.6678606176646156
WOLF (strong): th(805.8313947244413)=-34.63749140178175; dx=0.014183176953765642 evalInputDelta=1.5953873778621386
WOLF (strong): th(161.16627894488826)=-34.50579384888596; dx=0.0695639079662822 evalInputDelta=1.4636898249663446
New Minimum: -34.70996464158423 > -36.392522137741175
WOLF (strong): th(26.86104649081471)=-36.392522137741175; dx=0.38488015347280535 evalInputDelta=3.3504181138215614
New Minimum: -36.392522137741175 > -42.33624575377488
WOLF (strong): th(3.837292355830673)=-42.33624575377488; dx=1.9386619213526384 evalInputDelta=9.29414172985527
New Minimum: -42.33624575377488 > -51.148406770231105
END: th(0.47966154447883413)=-51.148406770231105; dx=-19.21297924572474 evalInputDelta=18.106302746311492
Fitness changed from -33.04210402391961 to -51.148406770231105
Iteration 14 complete. Error: -51.148406770231105 Total: 1.1833; Orientation: 0.0280; Line Search: 1.0443
Adding measurement 231a1e86 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 3.998e+04, gradient 2.004e-02, dot -0.207; [ba8c9e13-fdc0-4ece-8d28-68159c231c41 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -51.148406770231105, -33.04210402391961, -7.108732655829049, -4.517519505613512, -2.404648226291845
Rejected: LBFGS Orientation magnitude: 5.525e+04, gradient 2.004e-02, dot -0.238; [ba8c9e13-fdc0-4ece-8d28-68159c231c41 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -51.148406770231105, -33.04210402391961, -7.108732655829049, -4.517519505613512
LBFGS Accumulation History: 3 points
Removed measurement 231a1e86 to history. Total: 4
Removed measurement 14d8cf88 to history. Total: 3
th(0)=-51.148406770231105;dx=-3.898357195830961E-4
New Minimum: -51.148406770231105 > -51.148939644039764
END: th(1.0333994708994716)=-51.148939644039764; dx=-3.431184654239389E-4 evalInputDelta=5.328738086589624E-4
Fitness changed from -51.148406770231105 to -51.148939644039764
Iteration 15 complete. Error: -51.148939644039764 Total: 0.4029; Orientation: 0.0654; Line Search: 0.2289
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -51.148939644039764 (> -Infinity) after 8.146s (< 5400.000s)
Returns
-51.148939644039764