Code from BasicOptimizer.scala:75 executed in 3777.23 seconds (39.056 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1195104140194900
Reset training subject: 1195158314872800
Adding measurement 76d8a94 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=1.261156060131331;dx=-3.3513827684037126E-7
New Minimum: 1.261156060131331 > 1.2611559426908374
WOLFE (weak): th(2.154434690031884)=1.2611559426908374; dx=-3.3008410746129216E-7 evalInputDelta=1.174404935522233E-7
New Minimum: 1.2611559426908374 > 1.2611554360101063
WOLFE (weak): th(4.308869380063768)=1.2611554360101063; dx=-3.3008428768388444E-7 evalInputDelta=6.241212247015682E-7
New Minimum: 1.2611554360101063 > 1.2611544964193555
WOLFE (weak): th(12.926608140191302)=1.2611544964193555; dx=-3.3008363909883204E-7 evalInputDelta=1.5637119754607909E-6
New Minimum: 1.2611544964193555 > 1.2611499336246659
WOLFE (weak): th(51.70643256076521)=1.2611499336246659; dx=-3.3008185525636475E-7 evalInputDelta=6.126506665093956E-6
New Minimum: 1.2611499336246659 > 1.2611264059509608
WOLFE (weak): th(258.53216280382605)=1.2611264059509608; dx=-3.300804875236205E-7 evalInputDelta=2.9654180370153327E-5
New Minimum: 1.2611264059509608 > 1.2609784319541295
WOLFE (weak): th(1551.1929768229563)=1.2609784319541295; dx=-3.3005787817728935E-7 evalInputDelta=1.7762817720146096E-4
New Minimum: 1.2609784319541295 > 1.25991361984347
WOLFE (weak): th(10858.350837760694)=1.25991361984347; dx=-3.297906108038289E-7 evalInputDelta=0.0012424402878610241
New Minimum: 1.25991361984347 > 1.2512644525261751
WOLFE (weak): th(86866.80670208555)=1.2512644525261751; dx=-3.278698558489628E-7 evalInputDelta=0.009891607605155839
New Minimum: 1.2512644525261751 > 1.1759727672089093
WOLFE (weak): th(781801.26031877)=1.1759727672089093; dx=-3.0980668662127166E-7 evalInputDelta=0.08518329292242166
New Minimum: 1.1759727672089093 > 0.7659715575802315
END: th(7818012.6031877)=0.7659715575802315; dx=-1.6087201217432335E-7 evalInputDelta=0.49518450255109947
Fitness changed from 1.261156060131331 to 0.7659715575802315
Iter
...skipping 11090 bytes...
history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.442e+04, gradient 1.418e-04, dot -0.867; [b2655d85-68bb-4b0f-88d8-410db39760bf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.03943956457585773, -0.09393500790882559, -0.14558973426953492, -0.18698503242883413
LBFGS Accumulation History: 3 points
Removed measurement 1847e32d to history. Total: 4
th(0)=-0.18698503242883413;dx=-2.0109205342704962E-8
Armijo: th(3.655251206470066E7)=-0.16450658382423466; dx=-8.21569772779996E-9 evalInputDelta=-0.022478448604599466
New Minimum: -0.18698503242883413 > -0.2215088672564669
END: th(1.827625603235033E7)=-0.2215088672564669; dx=-1.1321272687730637E-8 evalInputDelta=0.03452383482763277
Fitness changed from -0.18698503242883413 to -0.2215088672564669
Iteration 14 complete. Error: -0.2215088672564669 Total: 220.5607; Orientation: 1.4822; Line Search: 164.4602
Adding measurement 480842c9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.366e+03, gradient 1.374e-04, dot -0.800; [b2655d85-68bb-4b0f-88d8-410db39760bf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.09393500790882559, -0.14558973426953492, -0.18698503242883413, -0.2215088672564669
LBFGS Accumulation History: 3 points
Removed measurement 2a750d3a to history. Total: 4
th(0)=-0.2215088672564669;dx=-1.8878813637724574E-8
Armijo: th(3.937500000000003E7)=-0.10907720248913155; dx=6.883873056732585E-12 evalInputDelta=-0.11243166476733535
New Minimum: -0.2215088672564669 > -0.23441524864977403
END: th(1.9687500000000015E7)=-0.23441524864977403; dx=-5.630782886751537E-9 evalInputDelta=0.012906381393307131
Fitness changed from -0.2215088672564669 to -0.23441524864977403
Iteration 15 complete. Error: -0.23441524864977403 Total: 219.4527; Orientation: 1.4805; Line Search: 163.1827
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 15: -0.23441524864977403 (> -Infinity) after 3777.226s (< 3600.000s)
-0.23441524864977403