BasicOptimizer.scala:89 executed in 217.42 seconds (3.723 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 694498598670400
Reset training subject: 694503237000300
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -1079.032784488034 > -1079.6216930017663
th(0)=-1079.6216930017663;dx=-0.005714562068329425
New Minimum: -1079.6216930017663 > -1079.6338499695162
END: th(2.154434690031884)=-1079.6338499695162; dx=-0.005129048425120627 evalInputDelta=0.012156967749888281
Fitness changed from -1079.032784488034 to -1079.6338499695162
Iteration 1 complete. Error: -1079.6338499695162 Total: 22.8819; Orientation: 0.0646; Line Search: 9.0498
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 2930b1b3 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-1079.6338499695162;dx=-0.005143129396098792
New Minimum: -1079.6338499695162 > -1079.6600159309412
WOLFE (weak): th(4.641588833612779)=-1079.6600159309412; dx=-0.0051372419198908595 evalInputDelta=0.026165961425022033
New Minimum: -1079.6600159309412 > -1079.6861812582395
WOLFE (weak): th(9.283177667225559)=-1079.6861812582395; dx=-0.0051365111227626836 evalInputDelta=0.05233128872328052
New Minimum: -1079.6861812582395 > -1079.7908362270907
WOLFE (weak): th(27.849533001676676)=-1079.7908362270907; dx=-0.005130017305590075 evalInputDelta=0.15698625757454465
New Minimum: -1079.7908362270907 > -1080.2614266926316
WOLFE (weak): th(111.3981320067067)=-1080.2614266926316; dx=-0.005129168603729227 evalInputDelta=0.6275767231154532
New Minimum: -1080.2614266926316 > -1082.7683270210669
WOLFE (weak): th(556.9906600335335)=-1082.7683270210669; dx=-0.005122934067444599 evalInputDelta=3.1344770515506752
New Minimum: -1082.7683270210669 > -1098.2873599381903
WOLFE (weak): th(3341.9439602012008)=-1098.2873599381903; dx=-0.005039346006134199 evalInputDelta=18.653509968674143
New Minimum: -1098.2873599381903 > -1200.6611720214826
END: th(23393.607721408407)=-1200.6611720214826; dx=-0.004244057540699913 evalInputDelta=121.0273220519664
Fitness changed from -1079.6338499695162 to -1200.6611720214826
Iteration 2 complete. Error: -1200.6611720214826 Total: 42.4273; Orientation: 0.1027; Line Search: 36.8275
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 5a9a7050 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-1200.6611720214826;dx=-0.004070393292152506
New Minimum: -1200.6611720214826 > -1385.1227382437992
END: th(50400.00000000001)=-1385.1227382437992; dx=-0.0026802718589023855 evalInputDelta=184.4615662223166
Fitness changed from -1200.6611720214826 to -1385.1227382437992
Iteration 3 complete. Error: -1385.1227382437992 Total: 14.8237; Orientation: 0.0920; Line Search: 9.0151
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 64e8c487 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-1385.1227382437992;dx=-0.002677718652860962
New Minimum: -1385.1227382437992 > -1621.3124313345654
END: th(108583.50837760697)=-1621.3124313345654; dx=-0.0014263573382088082 evalInputDelta=236.18969309076624
Fitness changed from -1385.1227382437992 to -1621.3124313345654
Iteration 4 complete. Error: -1621.3124313345654 Total: 15.0772; Orientation: 0.0868; Line Search: 9.3558
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 33383fae to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.865e+04, gradient 4.688e-02, dot -0.682; [a0b5f4d5-094b-4e8d-867a-3f19d4ecf5ba = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1621.3124313345654, -1385.1227382437992, -1200.6611720214826, -1079.6338499695162
LBFGS Accumulation History: 3 points
Removed measurement 33383fae to history. Total: 3
th(0)=-1621.3124313345654;dx=-0.0021932885490335463
New Minimum: -1621.3124313345654 > -1863.3938123756877
END: th(233936.07721408413)=-1863.3938123756877; dx=-4.5726731721272616E-4 evalInputDelta=242.0813810411223
Fitness changed from -1621.3124313345654 to -1863.3938123756877
Iteration 5 complete. Error: -1863.3938123756877 Total: 16.5792; Orientation: 1.7497; Line Search: 9.1064
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 5eff58a6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.647e+04, gradient 5.397e-02, dot -0.424; [a0b5f4d5-094b-4e8d-867a-3f19d4ecf5ba = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1863.3938123756877, -1385.1227382437992, -1200.6611720214826, -1079.6338499695162
LBFGS Accumulation History: 3 points
Removed measurement 5eff58a6 to history. Total: 3
th(0)=-1863.3938123756877;dx=-0.0029105749795139994
New Minimum: -1863.3938123756877 > -1937.1050565211997
WOLF (strong): th(504000.0000000002)=-1937.1050565211997; dx=3.0587954922905845E-4 evalInputDelta=73.71124414551196
New Minimum: -1937.1050565211997 > -2028.396657461257
WOLF (strong): th(252000.0000000001)=-2028.396657461257; dx=1.9112417759757443E-4 evalInputDelta=165.00284508556933
END: th(84000.00000000003)=-2018.9712727030071; dx=-7.173537168432768E-4 evalInputDelta=155.5774603273194
Fitness changed from -1863.3938123756877 to -2028.396657461257
Iteration 6 complete. Error: -2028.396657461257 Total: 25.3233; Orientation: 1.6943; Line Search: 18.2673
Adding measurement 5a1a3eb0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.818e+04, gradient 5.583e-02, dot -0.327; [a0b5f4d5-094b-4e8d-867a-3f19d4ecf5ba = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2028.396657461257, -1385.1227382437992, -1200.6611720214826, -1079.6338499695162
LBFGS Accumulation History: 3 points
Removed measurement 5a1a3eb0 to history. Total: 3
th(0)=-2028.396657461257;dx=-0.0031136087493111248
New Minimum: -2028.396657461257 > -2260.231957122985
WOLF (strong): th(180972.5139626783)=-2260.231957122985; dx=1.0399977377311208E-4 evalInputDelta=231.83529966172796
END: th(90486.25698133915)=-2228.5204733185647; dx=-9.995269102517686E-4 evalInputDelta=200.1238158573076
Fitness changed from -2028.396657461257 to -2260.231957122985
Iteration 7 complete. Error: -2260.231957122985 Total: 20.1258; Orientation: 1.7515; Line Search: 14.1010
Adding measurement 6cb45504 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.672e+04, gradient 5.192e-02, dot -0.354; [a0b5f4d5-094b-4e8d-867a-3f19d4ecf5ba = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2260.231957122985, -1385.1227382437992, -1200.6611720214826, -1079.6338499695162
LBFGS Accumulation History: 3 points
Removed measurement 6cb45504 to history. Total: 3
th(0)=-2260.231957122985;dx=-0.002690638940687629
New Minimum: -2260.231957122985 > -2422.4536115269575
WOLF (strong): th(194946.7310117368)=-2422.4536115269575; dx=3.280097439409829E-4 evalInputDelta=162.22165440397248
New Minimum: -2422.4536115269575 > -2423.1191675194186
END: th(97473.3655058684)=-2423.1191675194186; dx=-5.725340934226011E-4 evalInputDelta=162.8872103964336
Fitness changed from -2260.231957122985 to -2423.1191675194186
Iteration 8 complete. Error: -2423.1191675194186 Total: 19.6544; Orientation: 1.6110; Line Search: 13.5795
Adding measurement 5315d9fe to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.785e+04, gradient 4.048e-02, dot -0.476; [a0b5f4d5-094b-4e8d-867a-3f19d4ecf5ba = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2423.1191675194186, -1385.1227382437992, -1200.6611720214826, -1079.6338499695162
LBFGS Accumulation History: 3 points
Removed measurement 5315d9fe to history. Total: 3
th(0)=-2423.1191675194186;dx=-0.0016368969575704234
New Minimum: -2423.1191675194186 > -2563.684067899163
END: th(210000.0000000001)=-2563.684067899163; dx=-1.5726842308746075E-4 evalInputDelta=140.56490037974436
Fitness changed from -2423.1191675194186 to -2563.684067899163
Iteration 9 complete. Error: -2563.684067899163 Total: 15.1986; Orientation: 1.6319; Line Search: 9.0477
Adding measurement 6d2fe436 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.370e+04, gradient 5.022e-02, dot -0.374; [a0b5f4d5-094b-4e8d-867a-3f19d4ecf5ba = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2563.684067899163, -1385.1227382437992, -1200.6611720214826, -1079.6338499695162
LBFGS Accumulation History: 3 points
Removed measurement 6d2fe436 to history. Total: 3
th(0)=-2563.684067899163;dx=-0.002517973162541331
Armijo: th(452431.2849066958)=-2485.4303450308157; dx=6.055558002223136E-4 evalInputDelta=-78.25372286834727
New Minimum: -2563.684067899163 > -2647.5985345214253
WOLF (strong): th(226215.6424533479)=-2647.5985345214253; dx=4.519104405627869E-4 evalInputDelta=83.91446662226235
New Minimum: -2647.5985345214253 > -2681.3440659490157
END: th(75405.21415111596)=-2681.3440659490157; dx=-5.167310408646557E-4 evalInputDelta=117.65999804985267
Fitness changed from -2563.684067899163 to -2681.3440659490157
Iteration 10 complete. Error: -2681.3440659490157 Total: 24.4070; Orientation: 1.6027; Line Search: 18.4963
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -2681.3440659490157 (> -Infinity) after 217.415s (< 5400.000s)
Returns
-2681.3440659490157