Code from BasicOptimizer.scala:75 executed in 930.49 seconds (7.241 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1025642117857200
Reset training subject: 1025649478291400
Adding measurement 58b1e9c7 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-1.1191013753414154;dx=-3.0853767490875387E-7
New Minimum: -1.1191013753414154 > -1.119102232158184
WOLFE (weak): th(2.154434690031884)=-1.119102232158184; dx=-3.0862102392777766E-7 evalInputDelta=8.568167686462402E-7
New Minimum: -1.119102232158184 > -1.119103156030178
WOLFE (weak): th(4.308869380063768)=-1.119103156030178; dx=-3.08566314791059E-7 evalInputDelta=1.780688762664795E-6
New Minimum: -1.119103156030178 > -1.1191064566373825
WOLFE (weak): th(12.926608140191302)=-1.1191064566373825; dx=-3.084836834258267E-7 evalInputDelta=5.081295967102051E-6
New Minimum: -1.1191064566373825 > -1.1191215217113495
WOLFE (weak): th(51.70643256076521)=-1.1191215217113495; dx=-3.08488704117178E-7 evalInputDelta=2.014636993408203E-5
New Minimum: -1.1191215217113495 > -1.1192015036940575
WOLFE (weak): th(258.53216280382605)=-1.1192015036940575; dx=-3.0861981548304916E-7 evalInputDelta=1.0012835264205933E-4
New Minimum: -1.1192015036940575 > -1.1197016537189484
WOLFE (weak): th(1551.1929768229563)=-1.1197016537189484; dx=-3.0845122970565663E-7 evalInputDelta=6.00278377532959E-4
New Minimum: -1.1197016537189484 > -1.123296394944191
WOLFE (weak): th(10858.350837760694)=-1.123296394944191; dx=-3.075183120661429E-7 evalInputDelta=0.004195019602775574
New Minimum: -1.123296394944191 > -1.1521945744752884
WOLFE (weak): th(86866.80670208555)=-1.1521945744752884; dx=-2.9909639110650736E-7 evalInputDelta=0.033093199133872986
New Minimum: -1.1521945744752884 > -1.3805020079016685
END: th(781801.26031877)=-1.3805020079016685; dx=-2.2917887065431892E-7 evalInputDelta=0.26140063256025314
Fitness changed from -1.1191013753414154 to -1.3805020079016685
Iteration 1 complete. Error: -1.3805020079016685 Total: 95.9090; Orientation: 0.0077; Line Search: 73.6214
<a id="p-3"></a>Iteration 1
<a id="p-2"></
...skipping 25017 bytes...
5 Total: 44.3593; Orientation: 0.2626; Line Search: 36.8210
Adding measurement 59b53dd7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.902e+03, gradient 6.599e-05, dot -0.627; [67413a49-b19a-43e1-a2cc-224dbbbb5590 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.6896405071020126, -2.694264456629753, -2.698495991528034, -2.7032518573105335
LBFGS Accumulation History: 3 points
Removed measurement 36720df to history. Total: 4
th(0)=-2.7032518573105335;dx=-4.344233011555703E-9
New Minimum: -2.7032518573105335 > -2.707645356655121
END: th(1958617.9732885736)=-2.707645356655121; dx=-1.907429386863344E-9 evalInputDelta=0.004393499344587326
Fitness changed from -2.7032518573105335 to -2.707645356655121
Iteration 29 complete. Error: -2.707645356655121 Total: 22.7286; Orientation: 0.2509; Line Search: 14.8843
Adding measurement 783fac60 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.511e+03, gradient 4.456e-05, dot -0.920; [67413a49-b19a-43e1-a2cc-224dbbbb5590 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.694264456629753, -2.698495991528034, -2.7032518573105335, -2.707645356655121
LBFGS Accumulation History: 3 points
Removed measurement 2342685b to history. Total: 4
th(0)=-2.707645356655121;dx=-1.9803673865677787E-9
New Minimum: -2.707645356655121 > -2.7112215608358383
WOLFE (weak): th(4219714.506172844)=-2.7112215608358383; dx=-1.8392946408617087E-9 evalInputDelta=0.0035762041807174683
New Minimum: -2.7112215608358383 > -2.7140826359391212
END: th(8439429.012345688)=-2.7140826359391212; dx=-1.7124119408240496E-9 evalInputDelta=0.006437279284000397
Fitness changed from -2.707645356655121 to -2.7140826359391212
Iteration 30 complete. Error: -2.7140826359391212 Total: 30.0065; Orientation: 0.2686; Line Search: 22.3937
<a id="p-21"></a>Iteration 30
<a id="p-20"></a>
Final threshold in iteration 31: -2.7140826359391212 (> -Infinity) after 930.489s (< 3600.000s)
-2.7140826359391212