BasicOptimizer.scala:89 executed in 84.41 seconds (0.923 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 305405134901600
Reset training subject: 305406871702600
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -145.51481734765323 > -145.5604669753985
th(0)=-145.5604669753985;dx=-3.544056207274872E-5
New Minimum: -145.5604669753985 > -145.56053060336262
END: th(2.154434690031884)=-145.56053060336262; dx=-2.6106508642802225E-5 evalInputDelta=6.362796412417993E-5
Fitness changed from -145.51481734765323 to -145.56053060336262
Iteration 1 complete. Error: -145.56053060336262 Total: 8.6182; Orientation: 0.0383; Line Search: 3.5685
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 529515d0 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-145.56053060336262;dx=-2.6074966246943054E-5
New Minimum: -145.56053060336262 > -145.56065785920748
WOLFE (weak): th(4.641588833612779)=-145.56065785920748; dx=-2.603966931065967E-5 evalInputDelta=1.2725584485906438E-4
New Minimum: -145.56065785920748 > -145.56079925446016
WOLFE (weak): th(9.283177667225559)=-145.56079925446016; dx=-2.6016802304653275E-5 evalInputDelta=2.686510975422607E-4
New Minimum: -145.56079925446016 > -145.56132948543453
WOLFE (weak): th(27.849533001676676)=-145.56132948543453; dx=-2.6003949459916218E-5 evalInputDelta=7.988820719049272E-4
New Minimum: -145.56132948543453 > -145.56372610534362
WOLFE (weak): th(111.3981320067067)=-145.56372610534362; dx=-2.60146275536582E-5 evalInputDelta=0.003195501980997051
New Minimum: -145.56372610534362 > -145.57652154982554
WOLFE (weak): th(556.9906600335335)=-145.57652154982554; dx=-2.598905563140035E-5 evalInputDelta=0.015990946462920874
New Minimum: -145.57652154982554 > -145.65645701368067
WOLFE (weak): th(3341.9439602012008)=-145.65645701368067; dx=-2.5993257446437002E-5 evalInputDelta=0.09592641031804305
New Minimum: -145.65645701368067 > -146.23123588871772
WOLFE (weak): th(23393.607721408407)=-146.23123588871772; dx=-2.5931318097897198E-5 evalInputDelta=0.6707052853550977
New Minimum: -146.23123588871772 > -150.84481561274367
WOLFE (weak): th(187148.86177126726)=-150.84481561274367; dx=-2.50945297234032E-5 evalInputDelta=5.284285009381051
New Minimum: -150.84481561274367 > -183.73234221321152
END: th(1684339.7559414054)=-183.73234221321152; dx=-1.449615108416971E-5 evalInputDelta=38.1718116098489
Fitness changed from -145.56053060336262 to -183.73234221321152
Iteration 2 complete. Error: -183.73234221321152 Total: 18.2442; Orientation: 0.0571; Line Search: 16.3045
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 6df0a76b to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-183.73234221321152;dx=-1.7022164964450073E-5
New Minimum: -183.73234221321152 > -224.55899437098034
END: th(3628800.0000000005)=-224.55899437098034; dx=-5.163246239188165E-6 evalInputDelta=40.82665215776882
Fitness changed from -183.73234221321152 to -224.55899437098034
Iteration 3 complete. Error: -224.55899437098034 Total: 5.0396; Orientation: 0.0553; Line Search: 3.1332
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 784be1dc to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-224.55899437098034;dx=-1.0694536046032266E-5
New Minimum: -224.55899437098034 > -251.70924771609396
WOLF (strong): th(7818012.603187701)=-251.70924771609396; dx=6.052613989520808E-7 evalInputDelta=27.15025334511361
END: th(3909006.3015938504)=-249.4418808783414; dx=-2.433801859089214E-6 evalInputDelta=24.882886507361064
Fitness changed from -224.55899437098034 to -251.70924771609396
Iteration 4 complete. Error: -251.70924771609396 Total: 6.8945; Orientation: 0.0510; Line Search: 4.8460
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 7f939f1e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.912e+04, gradient 3.866e-03, dot -0.405; [a0c1d8c3-2517-401a-93a4-4ce9562c352c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -251.70924771609396, -224.55899437098034, -183.73234221321152, -145.56053060336262
LBFGS Accumulation History: 3 points
Removed measurement 7f939f1e to history. Total: 3
th(0)=-251.70924771609396;dx=-1.4926135630331462E-5
New Minimum: -251.70924771609396 > -255.88417878462343
WOLF (strong): th(8421698.779707028)=-255.88417878462343; dx=5.7698255419848355E-6 evalInputDelta=4.1749310685294745
New Minimum: -255.88417878462343 > -278.9024374830195
WOLF (strong): th(4210849.389853514)=-278.9024374830195; dx=1.8107756808795331E-6 evalInputDelta=27.19318976692557
END: th(1403616.4632845046)=-270.0824751069329; dx=-8.809831231486308E-6 evalInputDelta=18.373227390838963
Fitness changed from -251.70924771609396 to -278.9024374830195
Iteration 5 complete. Error: -278.9024374830195 Total: 9.7844; Orientation: 1.0938; Line Search: 6.7865
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 733a4232 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.865e+04, gradient 3.016e-03, dot -0.409; [a0c1d8c3-2517-401a-93a4-4ce9562c352c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -278.9024374830195, -224.55899437098034, -183.73234221321152, -145.56053060336262
LBFGS Accumulation History: 3 points
Removed measurement 733a4232 to history. Total: 3
th(0)=-278.9024374830195;dx=-9.082382137133637E-6
New Minimum: -278.9024374830195 > -295.2211479214696
END: th(3024000.0000000005)=-295.2211479214696; dx=-1.1058478704427332E-6 evalInputDelta=16.31871043845007
Fitness changed from -278.9024374830195 to -295.2211479214696
Iteration 6 complete. Error: -295.2211479214696 Total: 6.4765; Orientation: 1.0772; Line Search: 3.5423
Adding measurement 676b72d1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.949e+04, gradient 2.169e-03, dot -0.525; [a0c1d8c3-2517-401a-93a4-4ce9562c352c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -295.2211479214696, -224.55899437098034, -183.73234221321152, -145.56053060336262
LBFGS Accumulation History: 3 points
Removed measurement 676b72d1 to history. Total: 3
th(0)=-295.2211479214696;dx=-4.7035780101713995E-6
New Minimum: -295.2211479214696 > -306.17512853399904
WOLF (strong): th(6515010.502656418)=-306.17512853399904; dx=5.193114271056066E-7 evalInputDelta=10.953980612529449
END: th(3257505.251328209)=-305.4937107633661; dx=-1.450500579950316E-6 evalInputDelta=10.272562841896502
Fitness changed from -295.2211479214696 to -306.17512853399904
Iteration 7 complete. Error: -306.17512853399904 Total: 7.5625; Orientation: 1.0426; Line Search: 4.8558
Adding measurement 6530c27b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.377e+04, gradient 2.926e-03, dot -0.367; [a0c1d8c3-2517-401a-93a4-4ce9562c352c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -306.17512853399904, -224.55899437098034, -183.73234221321152, -145.56053060336262
LBFGS Accumulation History: 3 points
Removed measurement 6530c27b to history. Total: 3
th(0)=-306.17512853399904;dx=-8.551465933020891E-6
Armijo: th(7018082.316422524)=-304.60771577779104; dx=4.653677503355932E-6 evalInputDelta=-1.5674127562080002
New Minimum: -306.17512853399904 > -319.12725880469065
WOLF (strong): th(3509041.158211262)=-319.12725880469065; dx=1.3132605333206635E-6 evalInputDelta=12.952130270691612
END: th(1169680.3860704207)=-314.9585761743227; dx=-5.1951357283974275E-6 evalInputDelta=8.78344764032363
Fitness changed from -306.17512853399904 to -319.12725880469065
Iteration 8 complete. Error: -319.12725880469065 Total: 9.1208; Orientation: 0.9928; Line Search: 6.4432
Adding measurement 66425e80 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.947e+04, gradient 2.198e-03, dot -0.436; [a0c1d8c3-2517-401a-93a4-4ce9562c352c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -319.12725880469065, -224.55899437098034, -183.73234221321152, -145.56053060336262
LBFGS Accumulation History: 3 points
Removed measurement 66425e80 to history. Total: 3
th(0)=-319.12725880469065;dx=-4.826552090544134E-6
New Minimum: -319.12725880469065 > -327.0427059278203
END: th(2520000.000000001)=-327.0427059278203; dx=-1.1587558529532358E-6 evalInputDelta=7.915447123129638
Fitness changed from -319.12725880469065 to -327.0427059278203
Iteration 9 complete. Error: -327.0427059278203 Total: 6.1290; Orientation: 1.0177; Line Search: 3.4696
Adding measurement 2fda41a5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.991e+04, gradient 1.578e-03, dot -0.548; [a0c1d8c3-2517-401a-93a4-4ce9562c352c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -327.0427059278203, -224.55899437098034, -183.73234221321152, -145.56053060336262
LBFGS Accumulation History: 3 points
Removed measurement 2fda41a5 to history. Total: 3
th(0)=-327.0427059278203;dx=-2.48650962711165E-6
New Minimum: -327.0427059278203 > -334.8243359491729
END: th(5429175.418880349)=-334.8243359491729; dx=-5.904638226586087E-7 evalInputDelta=7.781630021352612
Fitness changed from -327.0427059278203 to -334.8243359491729
Iteration 10 complete. Error: -334.8243359491729 Total: 6.2591; Orientation: 1.1120; Line Search: 3.5294
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -334.8243359491729 (> -Infinity) after 84.408s (< 5400.000s)
Returns
-334.8243359491729