BasicOptimizer.scala:89 executed in 59.81 seconds (1.671 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 149842769672800
Reset training subject: 149843741468000
Adding measurement 7c37b396 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-2.602894101974777;dx=-1.73411847128762E-4
New Minimum: -2.602894101974777 > -2.603335934217078
WOLFE (weak): th(2.154434690031884)=-2.603335934217078; dx=-1.7382766881198235E-4 evalInputDelta=4.418322423007659E-4
New Minimum: -2.603335934217078 > -2.6037774505034452
WOLFE (weak): th(4.308869380063768)=-2.6037774505034452; dx=-1.7298753958477664E-4 evalInputDelta=8.833485286681153E-4
New Minimum: -2.6037774505034452 > -2.6055451756729475
WOLFE (weak): th(12.926608140191302)=-2.6055451756729475; dx=-1.7384051400920903E-4 evalInputDelta=0.002651073698170414
New Minimum: -2.6055451756729475 > -2.6135039893702277
WOLFE (weak): th(51.70643256076521)=-2.6135039893702277; dx=-1.7412197096621792E-4 evalInputDelta=0.010609887395450546
New Minimum: -2.6135039893702277 > -2.6560465308433527
WOLFE (weak): th(258.53216280382605)=-2.6560465308433527; dx=-1.746481754301665E-4 evalInputDelta=0.05315242886857563
New Minimum: -2.6560465308433527 > -2.926458074233226
WOLFE (weak): th(1551.1929768229563)=-2.926458074233226; dx=-1.7896767579083902E-4 evalInputDelta=0.3235639722584489
New Minimum: -2.926458074233226 > -4.8465523170055
END: th(10858.350837760694)=-4.8465523170055; dx=-1.5454079957649754E-4 evalInputDelta=2.243658215030723
Fitness changed from -2.602894101974777 to -4.8465523170055
Iteration 1 complete. Error: -4.8465523170055 Total: 9.7748; Orientation: 0.0143; Line Search: 7.2324
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 7ce45a68 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-4.8465523170055;dx=-2.2188130194952034E-4
New Minimum: -4.8465523170055 > -9.49841509813642
END: th(23393.607721408407)=-9.49841509813642; dx=-1.1076722700953674E-4 evalInputDelta=4.6518627811309194
Fitness changed from -4.8465523170055 to -9.49841509813642
Iteration 2 complete. Error: -9.49841509813642 Total: 2.6740; Orientation: 0.0149; Line Search: 1.7569
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 1497948d to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-9.49841509813642;dx=-3.1847631338296096E-4
New Minimum: -9.49841509813642 > -13.151704982786944
END: th(50400.00000000001)=-13.151704982786944; dx=-5.248107853897583E-6 evalInputDelta=3.6532898846505244
Fitness changed from -9.49841509813642 to -13.151704982786944
Iteration 3 complete. Error: -13.151704982786944 Total: 2.6157; Orientation: 0.0147; Line Search: 1.7112
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 35eab565 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.470e+04, gradient 2.352e-02, dot -0.158; [fd0b8d39-198f-455d-b084-4b2ecab679fa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -13.151704982786944, -9.49841509813642, -4.8465523170055, -2.602894101974777
LBFGS Accumulation History: 3 points
Removed measurement 35eab565 to history. Total: 3
th(0)=-13.151704982786944;dx=-5.499175081603849E-4
New Minimum: -13.151704982786944 > -15.466400138187451
WOLF (strong): th(108583.50837760697)=-15.466400138187451; dx=1.214389463001507E-5 evalInputDelta=2.314695155400507
New Minimum: -15.466400138187451 > -17.92491250804648
WOLF (strong): th(54291.75418880348)=-17.92491250804648; dx=3.553377492607588E-5 evalInputDelta=4.773207525259535
New Minimum: -17.92491250804648 > -19.65496427369908
END: th(18097.251396267828)=-19.65496427369908; dx=-1.120417916931813E-4 evalInputDelta=6.503259290912135
Fitness changed from -13.151704982786944 to -19.65496427369908
Iteration 4 complete. Error: -19.65496427369908 Total: 4.6687; Orientation: 0.3623; Line Search: 3.4167
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 67b8441d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.698e+04, gradient 1.921e-02, dot -0.182; [fd0b8d39-198f-455d-b084-4b2ecab679fa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.65496427369908, -9.49841509813642, -4.8465523170055, -2.602894101974777
LBFGS Accumulation History: 3 points
Removed measurement 67b8441d to history. Total: 3
th(0)=-19.65496427369908;dx=-3.683272971444479E-4
New Minimum: -19.65496427369908 > -24.17193284817609
WOLF (strong): th(38989.346202347355)=-24.17193284817609; dx=6.154715946319797E-6 evalInputDelta=4.516968574477012
New Minimum: -24.17193284817609 > -24.253832424519135
END: th(19494.673101173677)=-24.253832424519135; dx=-1.0644517317072766E-4 evalInputDelta=4.598868150820056
Fitness changed from -19.65496427369908 to -24.253832424519135
Iteration 5 complete. Error: -24.253832424519135 Total: 3.8213; Orientation: 0.2779; Line Search: 2.5813
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 3a002762 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.820e+04, gradient 1.766e-02, dot -0.183; [fd0b8d39-198f-455d-b084-4b2ecab679fa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -24.253832424519135, -9.49841509813642, -4.8465523170055, -2.602894101974777
LBFGS Accumulation History: 3 points
Removed measurement 3a002762 to history. Total: 3
th(0)=-24.253832424519135;dx=-3.100921390891781E-4
New Minimum: -24.253832424519135 > -28.176583165647862
END: th(42000.000000000015)=-28.176583165647862; dx=-1.5021882942946984E-5 evalInputDelta=3.922750741128727
Fitness changed from -24.253832424519135 to -28.176583165647862
Iteration 6 complete. Error: -28.176583165647862 Total: 3.0609; Orientation: 0.2826; Line Search: 1.8342
Adding measurement 466e0d8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.730e+04, gradient 2.179e-02, dot -0.143; [fd0b8d39-198f-455d-b084-4b2ecab679fa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -28.176583165647862, -9.49841509813642, -4.8465523170055, -2.602894101974777
LBFGS Accumulation History: 3 points
Removed measurement 466e0d8 to history. Total: 3
th(0)=-28.176583165647862;dx=-4.7328183992479983E-4
Armijo: th(90486.25698133915)=-24.339121951457233; dx=4.262618672718272E-5 evalInputDelta=-3.837461214190629
New Minimum: -28.176583165647862 > -30.732324375877603
WOLF (strong): th(45243.12849066957)=-30.732324375877603; dx=9.235882360397759E-5 evalInputDelta=2.5557412102297405
New Minimum: -30.732324375877603 > -33.53138702032892
END: th(15081.04283022319)=-33.53138702032892; dx=-1.3818096924775753E-4 evalInputDelta=5.354803854681059
Fitness changed from -28.176583165647862 to -33.53138702032892
Iteration 7 complete. Error: -33.53138702032892 Total: 4.4959; Orientation: 0.2750; Line Search: 3.3838
Adding measurement 43125895 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.384e+04, gradient 1.545e-02, dot -0.206; [fd0b8d39-198f-455d-b084-4b2ecab679fa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -33.53138702032892, -9.49841509813642, -4.8465523170055, -2.602894101974777
LBFGS Accumulation History: 3 points
Removed measurement 43125895 to history. Total: 3
th(0)=-33.53138702032892;dx=-2.3719386628802007E-4
New Minimum: -33.53138702032892 > -37.84756601117816
END: th(32491.12183528946)=-37.84756601117816; dx=-5.5295300631758764E-5 evalInputDelta=4.316178990849238
Fitness changed from -33.53138702032892 to -37.84756601117816
Iteration 8 complete. Error: -37.84756601117816 Total: 2.7811; Orientation: 0.2787; Line Search: 1.7065
Adding measurement 4331b4a5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.277e+04, gradient 1.703e-02, dot -0.211; [fd0b8d39-198f-455d-b084-4b2ecab679fa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -37.84756601117816, -9.49841509813642, -4.8465523170055, -2.602894101974777
LBFGS Accumulation History: 3 points
Removed measurement 4331b4a5 to history. Total: 3
th(0)=-37.84756601117816;dx=-2.876144637905393E-4
Armijo: th(70000.00000000003)=-36.24726808489481; dx=2.22722049528207E-5 evalInputDelta=-1.6002979262833463
New Minimum: -37.84756601117816 > -39.77377741296051
WOLF (strong): th(35000.000000000015)=-39.77377741296051; dx=3.1440023538459346E-5 evalInputDelta=1.9262114017823535
New Minimum: -39.77377741296051 > -40.47756384024906
END: th(11666.666666666672)=-40.47756384024906; dx=-1.256255925666703E-4 evalInputDelta=2.629997829070902
Fitness changed from -37.84756601117816 to -40.47756384024906
Iteration 9 complete. Error: -40.47756384024906 Total: 4.5786; Orientation: 0.2793; Line Search: 3.5135
Adding measurement 38517f6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.480e+04, gradient 1.305e-02, dot -0.250; [fd0b8d39-198f-455d-b084-4b2ecab679fa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -40.47756384024906, -9.49841509813642, -4.8465523170055, -2.602894101974777
LBFGS Accumulation History: 3 points
Removed measurement 38517f6 to history. Total: 3
th(0)=-40.47756384024906;dx=-1.6832578091708646E-4
New Minimum: -40.47756384024906 > -43.343162349260304
END: th(25135.071383705323)=-43.343162349260304; dx=-9.374099847895531E-5 evalInputDelta=2.8655985090112424
Fitness changed from -40.47756384024906 to -43.343162349260304
Iteration 10 complete. Error: -43.343162349260304 Total: 2.7701; Orientation: 0.2781; Line Search: 1.7155
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 51dd6ba to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.312e+04, gradient 1.417e-02, dot -0.268; [fd0b8d39-198f-455d-b084-4b2ecab679fa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -43.343162349260304, -9.49841509813642, -4.8465523170055, -2.602894101974777
LBFGS Accumulation History: 3 points
Removed measurement 51dd6ba to history. Total: 3
th(0)=-43.343162349260304;dx=-2.0000007069940988E-4
New Minimum: -43.343162349260304 > -45.133656916223735
END: th(54151.86972548245)=-45.133656916223735; dx=-2.7825312158398975E-5 evalInputDelta=1.7904945669634316
Fitness changed from -43.343162349260304 to -45.133656916223735
Iteration 11 complete. Error: -45.133656916223735 Total: 2.8604; Orientation: 0.2869; Line Search: 1.6948
Adding measurement 115b0846 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.765e+04, gradient 1.928e-02, dot -0.191; [fd0b8d39-198f-455d-b084-4b2ecab679fa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -45.133656916223735, -9.49841509813642, -4.8465523170055, -2.602894101974777
LBFGS Accumulation History: 3 points
Removed measurement 115b0846 to history. Total: 3
th(0)=-45.133656916223735;dx=-3.6908483939292826E-4
Armijo: th(116666.66666666674)=-40.476988420205224; dx=2.127640288140808E-5 evalInputDelta=-4.656668496018511
New Minimum: -45.133656916223735 > -45.85738487002294
WOLF (strong): th(58333.33333333337)=-45.85738487002294; dx=4.648796183181132E-5 evalInputDelta=0.7237279537992052
New Minimum: -45.85738487002294 > -49.611429203495305
END: th(19444.444444444456)=-49.611429203495305; dx=-6.938989022039E-5 evalInputDelta=4.47777228727157
Fitness changed from -45.133656916223735 to -49.611429203495305
Iteration 12 complete. Error: -49.611429203495305 Total: 5.3534; Orientation: 0.2839; Line Search: 4.2431
Adding measurement 68c2a5e2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.634e+04, gradient 1.488e-02, dot -0.233; [fd0b8d39-198f-455d-b084-4b2ecab679fa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -49.611429203495305, -9.49841509813642, -4.8465523170055, -2.602894101974777
LBFGS Accumulation History: 3 points
Removed measurement 68c2a5e2 to history. Total: 3
th(0)=-49.611429203495305;dx=-2.196462791191087E-4
New Minimum: -49.611429203495305 > -53.11907714346738
END: th(41891.785639508875)=-53.11907714346738; dx=-7.598833655244166E-6 evalInputDelta=3.507647939972074
Fitness changed from -49.611429203495305 to -53.11907714346738
Iteration 13 complete. Error: -53.11907714346738 Total: 2.7933; Orientation: 0.2803; Line Search: 1.7023
Adding measurement 2517f9b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.358e+04, gradient 1.871e-02, dot -0.216; [fd0b8d39-198f-455d-b084-4b2ecab679fa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -53.11907714346738, -9.49841509813642, -4.8465523170055, -2.602894101974777
LBFGS Accumulation History: 3 points
Removed measurement 2517f9b to history. Total: 3
th(0)=-53.11907714346738;dx=-3.4901986412649323E-4
Armijo: th(90253.11620913743)=-48.43837630394621; dx=3.855521535515225E-5 evalInputDelta=-4.680700839521172
New Minimum: -53.11907714346738 > -54.5746157537806
WOLF (strong): th(45126.558104568714)=-54.5746157537806; dx=6.472593892688759E-5 evalInputDelta=1.4555386103132193
New Minimum: -54.5746157537806 > -56.9401618142111
END: th(15042.186034856239)=-56.9401618142111; dx=-1.0654199077768931E-4 evalInputDelta=3.8210846707437227
Fitness changed from -53.11907714346738 to -56.9401618142111
Iteration 14 complete. Error: -56.9401618142111 Total: 4.5560; Orientation: 0.2769; Line Search: 3.4712
Adding measurement 12b77743 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.412e+05, gradient 1.238e-02, dot -0.306; [fd0b8d39-198f-455d-b084-4b2ecab679fa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -56.9401618142111, -9.49841509813642, -4.8465523170055, -2.602894101974777
LBFGS Accumulation History: 3 points
Removed measurement 12b77743 to history. Total: 3
th(0)=-56.9401618142111;dx=-1.5204760344300305E-4
New Minimum: -56.9401618142111 > -60.360644229512374
END: th(32407.40740740743)=-60.360644229512374; dx=-8.559336863573307E-5 evalInputDelta=3.4204824153012723
Fitness changed from -56.9401618142111 to -60.360644229512374
Iteration 15 complete. Error: -60.360644229512374 Total: 2.9047; Orientation: 0.2716; Line Search: 1.7221
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -60.360644229512374 (> -Infinity) after 59.807s (< 5400.000s)
Returns
-60.360644229512374