BasicOptimizer.scala:89 executed in 314.21 seconds (3.915 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 646692844076600
Reset training subject: 646698884518700
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -37.64690817250428 > -37.741289937320964
th(0)=-37.741289937320964;dx=-2.0128639468038102E-6
New Minimum: -37.741289937320964 > -37.741293939687935
WOLFE (weak): th(2.154434690031884)=-37.741293939687935; dx=-1.934562379645583E-6 evalInputDelta=4.002366971178617E-6
New Minimum: -37.741293939687935 > -37.74129794205448
WOLFE (weak): th(4.308869380063768)=-37.74129794205448; dx=-1.9343605830509367E-6 evalInputDelta=8.004733516031592E-6
New Minimum: -37.74129794205448 > -37.74131795388085
WOLFE (weak): th(12.926608140191302)=-37.74131795388085; dx=-1.934453972583955E-6 evalInputDelta=2.8016559888044412E-5
New Minimum: -37.74131795388085 > -37.74140000225804
WOLFE (weak): th(51.70643256076521)=-37.74140000225804; dx=-1.934377297993731E-6 evalInputDelta=1.100649370755491E-4
New Minimum: -37.74140000225804 > -37.74184025879657
WOLFE (weak): th(258.53216280382605)=-37.74184025879657; dx=-1.934490349032034E-6 evalInputDelta=5.503214756075181E-4
New Minimum: -37.74184025879657 > -37.744593746826624
WOLFE (weak): th(1551.1929768229563)=-37.744593746826624; dx=-1.9342212647932304E-6 evalInputDelta=0.0033038095056596717
New Minimum: -37.744593746826624 > -37.76441253412847
WOLFE (weak): th(10858.350837760694)=-37.76441253412847; dx=-1.9329721520240106E-6 evalInputDelta=0.023122596807503726
New Minimum: -37.76441253412847 > -37.925893910925105
WOLFE (weak): th(86866.80670208555)=-37.925893910925105; dx=-1.9256744478497047E-6 evalInputDelta=0.1846039736041405
New Minimum: -37.925893910925105 > -39.367687167201346
WOLFE (weak): th(781801.26031877)=-39.367687167201346; dx=-1.8426680671859848E-6 evalInputDelta=1.6263972298803822
New Minimum: -39.367687167201346 > -49.444078905853395
END: th(7818012.6031877)=-49.444078905853395; dx=-7.725987883603656E-7 evalInputDelta=11.702788968532431
Fitness changed from -37.64690817250428 to -49.444078905853395
Iteration 1 complete. Error: -49.444078905853395 Total: 83.8320; Orientation: 0.0897; Line Search: 65.7346
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 4567cc74 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-49.444078905853395;dx=-1.0265353763552872E-6
New Minimum: -49.444078905853395 > -59.96574142114827
END: th(1.684339755941405E7)=-59.96574142114827; dx=-2.6917135778109405E-7 evalInputDelta=10.521662515294878
Fitness changed from -49.444078905853395 to -59.96574142114827
Iteration 2 complete. Error: -59.96574142114827 Total: 18.8927; Orientation: 0.1394; Line Search: 11.8012
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 6a393748 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-59.96574142114827;dx=-9.808554363712244E-7
New Minimum: -59.96574142114827 > -64.64883592172079
WOLF (strong): th(3.6288E7)=-64.64883592172079; dx=1.2305433311395743E-7 evalInputDelta=4.6830945005725155
New Minimum: -64.64883592172079 > -67.21718809141
WOLF (strong): th(1.8144E7)=-67.21718809141; dx=4.218556309106975E-8 evalInputDelta=7.251446670261721
END: th(6048000.0)=-65.11507690877924; dx=-5.386221100780908E-7 evalInputDelta=5.149335487630971
Fitness changed from -59.96574142114827 to -67.21718809141
Iteration 3 complete. Error: -67.21718809141 Total: 31.2073; Orientation: 0.1346; Line Search: 23.8963
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 6043f09 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-67.21718809141;dx=-1.0298408955286846E-6
New Minimum: -67.21718809141 > -75.18439087219473
END: th(1.3030021005312834E7)=-75.18439087219473; dx=-4.572715973442318E-8 evalInputDelta=7.967202780784731
Fitness changed from -67.21718809141 to -75.18439087219473
Iteration 4 complete. Error: -75.18439087219473 Total: 19.0495; Orientation: 0.1423; Line Search: 12.1675
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 42ed9d38 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.488e+04, gradient 7.712e-04, dot -0.690; [30ed9f96-6d49-4e01-aeaf-95a555b18b22 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -75.18439087219473, -67.21718809141, -59.96574142114827, -49.444078905853395
LBFGS Accumulation History: 3 points
Removed measurement 42ed9d38 to history. Total: 3
th(0)=-75.18439087219473;dx=-5.947188204260678E-7
New Minimum: -75.18439087219473 > -79.59735755354285
WOLF (strong): th(2.8072329265690092E7)=-79.59735755354285; dx=9.823416598330855E-8 evalInputDelta=4.412966681348124
New Minimum: -79.59735755354285 > -80.19531878984455
END: th(1.4036164632845046E7)=-80.19531878984455; dx=-9.600336607907575E-8 evalInputDelta=5.010927917649823
Fitness changed from -75.18439087219473 to -80.19531878984455
Iteration 5 complete. Error: -80.19531878984455 Total: 28.3065; Orientation: 2.7519; Line Search: 18.5819
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 1f02693f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.424e+04, gradient 7.394e-04, dot -0.684; [30ed9f96-6d49-4e01-aeaf-95a555b18b22 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -80.19531878984455, -67.21718809141, -59.96574142114827, -49.444078905853395
LBFGS Accumulation History: 3 points
Removed measurement 1f02693f to history. Total: 3
th(0)=-80.19531878984455;dx=-5.466106759161496E-7
New Minimum: -80.19531878984455 > -83.5066049125113
WOLF (strong): th(3.0240000000000007E7)=-83.5066049125113; dx=1.0354827616254736E-7 evalInputDelta=3.3112861226667576
New Minimum: -83.5066049125113 > -84.63554811055397
END: th(1.5120000000000004E7)=-84.63554811055397; dx=-4.031309973291391E-8 evalInputDelta=4.44022932070942
Fitness changed from -80.19531878984455 to -84.63554811055397
Iteration 6 complete. Error: -84.63554811055397 Total: 28.3319; Orientation: 2.9121; Line Search: 18.5523
Adding measurement 7e7243af to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.829e+04, gradient 7.409e-04, dot -0.628; [30ed9f96-6d49-4e01-aeaf-95a555b18b22 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -84.63554811055397, -67.21718809141, -59.96574142114827, -49.444078905853395
LBFGS Accumulation History: 3 points
Removed measurement 7e7243af to history. Total: 3
th(0)=-84.63554811055397;dx=-5.485646810556909E-7
New Minimum: -84.63554811055397 > -86.49288667644636
WOLF (strong): th(3.257505251328209E7)=-86.49288667644636; dx=1.5488555052847252E-7 evalInputDelta=1.857338565892391
New Minimum: -86.49288667644636 > -88.83100881760699
WOLF (strong): th(1.6287526256641045E7)=-88.83100881760699; dx=2.23632309748975E-8 evalInputDelta=4.195460707053016
END: th(5429175.418880348)=-87.33229087618028; dx=-3.445113501865728E-7 evalInputDelta=2.696742765626311
Fitness changed from -84.63554811055397 to -88.83100881760699
Iteration 7 complete. Error: -88.83100881760699 Total: 31.5803; Orientation: 2.5089; Line Search: 23.5052
Adding measurement 1a7982aa to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.080e+05, gradient 7.449e-04, dot -0.589; [30ed9f96-6d49-4e01-aeaf-95a555b18b22 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -88.83100881760699, -67.21718809141, -59.96574142114827, -49.444078905853395
LBFGS Accumulation History: 3 points
Removed measurement 1a7982aa to history. Total: 3
th(0)=-88.83100881760699;dx=-5.545787172675118E-7
New Minimum: -88.83100881760699 > -93.01213896435563
END: th(1.1696803860704206E7)=-93.01213896435563; dx=-8.519206565367286E-8 evalInputDelta=4.181130146748643
Fitness changed from -88.83100881760699 to -93.01213896435563
Iteration 8 complete. Error: -93.01213896435563 Total: 20.0025; Orientation: 2.5427; Line Search: 11.7469
Adding measurement 28077b8a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.426e+04, gradient 5.667e-04, dot -0.712; [30ed9f96-6d49-4e01-aeaf-95a555b18b22 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -93.01213896435563, -67.21718809141, -59.96574142114827, -49.444078905853395
LBFGS Accumulation History: 3 points
Removed measurement 28077b8a to history. Total: 3
th(0)=-93.01213896435563;dx=-3.210797986240044E-7
New Minimum: -93.01213896435563 > -97.0302805892823
END: th(2.5200000000000007E7)=-97.0302805892823; dx=-3.275083863878985E-8 evalInputDelta=4.01814162492667
Fitness changed from -93.01213896435563 to -97.0302805892823
Iteration 9 complete. Error: -97.0302805892823 Total: 19.9235; Orientation: 2.4141; Line Search: 11.7355
Adding measurement 476b758c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.119e+05, gradient 7.364e-04, dot -0.515; [30ed9f96-6d49-4e01-aeaf-95a555b18b22 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -97.0302805892823, -67.21718809141, -59.96574142114827, -49.444078905853395
LBFGS Accumulation History: 3 points
Removed measurement 476b758c to history. Total: 3
th(0)=-97.0302805892823;dx=-5.419991122130807E-7
Armijo: th(5.429175418880349E7)=-91.1774231478103; dx=2.2780287048744665E-7 evalInputDelta=-5.852857441471997
New Minimum: -97.0302805892823 > -98.94420528730265
WOLF (strong): th(2.7145877094401743E7)=-98.94420528730265; dx=2.154192363921001E-7 evalInputDelta=1.9139246980203524
New Minimum: -98.94420528730265 > -100.8179744789152
END: th(9048625.698133914)=-100.8179744789152; dx=-1.9092325570637478E-7 evalInputDelta=3.787693889632905
Fitness changed from -97.0302805892823 to -100.8179744789152
Iteration 10 complete. Error: -100.8179744789152 Total: 32.0111; Orientation: 2.4118; Line Search: 23.9478
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -100.8179744789152 (> -Infinity) after 314.212s (< 5400.000s)
Returns
-100.8179744789152