Code from BasicOptimizer.scala:75 executed in 567.75 seconds (5.313 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 5417522506641
Reset training subject: 5419853846341
Adding measurement 5b41e441 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=0.9778187629806805;dx=-2.2933897268965683E-8
New Minimum: 0.9778187629806805 > 0.9778187214157111
WOLFE (weak): th(2.154434690031884)=0.9778187214157111; dx=-2.293474193833128E-8 evalInputDelta=4.1564969355256665E-8
New Minimum: 0.9778187214157111 > 0.9778186798525184
WOLFE (weak): th(4.308869380063768)=0.9778186798525184; dx=-2.2934726038916884E-8 evalInputDelta=8.312816213162932E-8
New Minimum: 0.9778186798525184 > 0.9778185136048817
WOLFE (weak): th(12.926608140191302)=0.9778185136048817; dx=-2.293429310855981E-8 evalInputDelta=2.49375798788698E-7
New Minimum: 0.9778185136048817 > 0.9778177653739042
WOLFE (weak): th(51.70643256076521)=0.9778177653739042; dx=-2.29314683217234E-8 evalInputDelta=9.976067762984542E-7
New Minimum: 0.9778177653739042 > 0.9778137759444626
WOLFE (weak): th(258.53216280382605)=0.9778137759444626; dx=-2.2921904637788295E-8 evalInputDelta=4.987036217873886E-6
New Minimum: 0.9778137759444626 > 0.9777888848629781
WOLFE (weak): th(1551.1929768229563)=0.9777888848629781; dx=-2.2830967046917675E-8 evalInputDelta=2.9878117702386575E-5
New Minimum: 0.9777888848629781 > 0.9776120488367803
WOLFE (weak): th(10858.350837760694)=0.9776120488367803; dx=-2.2380216501928893E-8 evalInputDelta=2.067141439001663E-4
New Minimum: 0.9776120488367803 > 0.9762928139543146
END: th(86866.80670208555)=0.9762928139543146; dx=-1.8823377569797194E-8 evalInputDelta=0.0015259490263659359
Fitness changed from 0.9778187629806805 to 0.9762928139543146
Iteration 1 complete. Error: 0.9762928139543146 Total: 28.9006; Orientation: 0.0710; Line Search: 21.9101
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 58d6c6e4 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=0.9762928139543146;dx=-1.9001966720758962E-8
New Minimum: 0.97
...skipping 42545 bytes...
11.2594; Orientation: 1.6278; Line Search: 7.3337
Adding measurement 3e18d1c6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.726e+02, gradient 1.411e-04, dot -0.747; [b9faf838-c395-4712-ba6d-8ed5953036b3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.6854774119969853, 0.6815076776049739, 0.6809430443539516, 0.6796648651981801
LBFGS Accumulation History: 3 points
Removed measurement 39a7a822 to history. Total: 4
th(0)=0.6796648651981801;dx=-1.9897046270965414E-8
Armijo: th(2029727.177401608)=0.6836723438754001; dx=-2.3646731762167306E-9 evalInputDelta=-0.0040074786772199555
New Minimum: 0.6796648651981801 > 0.6789452952596791
END: th(1014863.588700804)=0.6789452952596791; dx=-6.7353354219699524E-9 evalInputDelta=7.195699385010856E-4
Fitness changed from 0.6796648651981801 to 0.6789452952596791
Iteration 49 complete. Error: 0.6789452952596791 Total: 11.2863; Orientation: 1.6252; Line Search: 7.3752
Adding measurement 642324c4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.665e+02, gradient 1.516e-04, dot -0.683; [b9faf838-c395-4712-ba6d-8ed5953036b3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.6815076776049739, 0.6809430443539516, 0.6796648651981801, 0.6789452952596791
LBFGS Accumulation History: 3 points
Removed measurement 5f1e7d4e to history. Total: 4
th(0)=0.6789452952596791;dx=-2.2977707000610444E-8
Armijo: th(2186457.321147262)=0.6861751850299586; dx=1.981225121109674E-9 evalInputDelta=-0.007229889770279496
New Minimum: 0.6789452952596791 > 0.677893995769048
END: th(1093228.660573631)=0.677893995769048; dx=-3.3342770851673894E-9 evalInputDelta=0.0010512994906310968
Fitness changed from 0.6789452952596791 to 0.677893995769048
Iteration 50 complete. Error: 0.677893995769048 Total: 11.3011; Orientation: 1.6302; Line Search: 7.3736
<a id="p-29"></a>Iteration 50
<a id="p-28"></a>
Final threshold in iteration 51: 0.677893995769048 (> -Infinity) after 567.748s (< 3600.000s)
0.677893995769048