BasicOptimizer.scala:89 executed in 294.74 seconds (3.448 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 646230191407800
Reset training subject: 646236068598700
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -35.111435115433835 > -35.158199484514824
th(0)=-35.158199484514824;dx=-1.849641202808571E-6
New Minimum: -35.158199484514824 > -35.158202706832
END: th(2.154434690031884)=-35.158202706832; dx=-1.6581732528968821E-6 evalInputDelta=3.2223171757550517E-6
Fitness changed from -35.111435115433835 to -35.158202706832
Iteration 1 complete. Error: -35.158202706832 Total: 29.6110; Orientation: 0.0964; Line Search: 12.3043
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 7ca35cb to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-35.158202706832;dx=-1.6636210457217107E-6
New Minimum: -35.158202706832 > -35.15821129967636
WOLFE (weak): th(4.641588833612779)=-35.15821129967636; dx=-1.6631299718859852E-6 evalInputDelta=8.592844359611718E-6
New Minimum: -35.15821129967636 > -35.15822096662375
WOLFE (weak): th(9.283177667225559)=-35.15822096662375; dx=-1.6630770248467813E-6 evalInputDelta=1.825979175151815E-5
New Minimum: -35.15822096662375 > -35.15825533797073
WOLFE (weak): th(27.849533001676676)=-35.15825533797073; dx=-1.6627577890222762E-6 evalInputDelta=5.263113872899794E-5
New Minimum: -35.15825533797073 > -35.158408934516956
WOLFE (weak): th(111.3981320067067)=-35.158408934516956; dx=-1.661953113519498E-6 evalInputDelta=2.0622768495570654E-4
New Minimum: -35.158408934516956 > -35.1592359813087
WOLFE (weak): th(556.9906600335335)=-35.1592359813087; dx=-1.661707777811593E-6 evalInputDelta=0.0010332744766969881
New Minimum: -35.1592359813087 > -35.164404046097054
WOLFE (weak): th(3341.9439602012008)=-35.164404046097054; dx=-1.6602897017334456E-6 evalInputDelta=0.006201339265054173
New Minimum: -35.164404046097054 > -35.20158163039387
WOLFE (weak): th(23393.607721408407)=-35.20158163039387; dx=-1.6593155529002434E-6 evalInputDelta=0.043378923561867566
New Minimum: -35.20158163039387 > -35.50330655536352
WOLFE (weak): th(187148.86177126726)=-35.50330655536352; dx=-1.6406941313045657E-6 evalInputDelta=0.3451038485315223
New Minimum: -35.50330655536352 > -38.07621083902471
END: th(1684339.7559414054)=-38.07621083902471; dx=-1.430152455612564E-6 evalInputDelta=2.9180081321927105
Fitness changed from -35.158202706832 to -38.07621083902471
Iteration 2 complete. Error: -38.07621083902471 Total: 67.0399; Orientation: 0.1429; Line Search: 59.9047
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 6ad9c63e to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-38.07621083902471;dx=-1.3683413304289202E-6
New Minimum: -38.07621083902471 > -42.67452601783015
END: th(3628800.0000000005)=-42.67452601783015; dx=-9.429880991009537E-7 evalInputDelta=4.598315178805443
Fitness changed from -38.07621083902471 to -42.67452601783015
Iteration 3 complete. Error: -42.67452601783015 Total: 18.5326; Orientation: 0.1322; Line Search: 11.6813
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 2d2226cd to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-42.67452601783015;dx=-9.18949036153033E-7
New Minimum: -42.67452601783015 > -48.99274375112233
END: th(7818012.603187701)=-48.99274375112233; dx=-5.603248388770548E-7 evalInputDelta=6.31821773329218
Fitness changed from -42.67452601783015 to -48.99274375112233
Iteration 4 complete. Error: -48.99274375112233 Total: 19.2453; Orientation: 0.1521; Line Search: 11.5076
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 2b83854a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.986e+04, gradient 8.540e-04, dot -0.751; [25b2f0ef-63b0-4491-af0b-2219757414d1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -48.99274375112233, -42.67452601783015, -38.07621083902471, -35.158202706832
LBFGS Accumulation History: 3 points
Removed measurement 2b83854a to history. Total: 3
th(0)=-48.99274375112233;dx=-7.282547393547871E-7
New Minimum: -48.99274375112233 > -56.586789786326875
END: th(1.6843397559414055E7)=-56.586789786326875; dx=-2.251911903025729E-7 evalInputDelta=7.594046035204542
Fitness changed from -48.99274375112233 to -56.586789786326875
Iteration 5 complete. Error: -56.586789786326875 Total: 21.3207; Orientation: 2.5619; Line Search: 11.8674
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 660527fe to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.045e+04, gradient 9.756e-04, dot -0.492; [25b2f0ef-63b0-4491-af0b-2219757414d1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -56.586789786326875, -42.67452601783015, -38.07621083902471, -35.158202706832
LBFGS Accumulation History: 3 points
Removed measurement 660527fe to history. Total: 3
th(0)=-56.586789786326875;dx=-9.509018756336604E-7
New Minimum: -56.586789786326875 > -58.24866551303876
WOLF (strong): th(3.628800000000001E7)=-58.24866551303876; dx=1.0889035135027202E-7 evalInputDelta=1.6618757267118838
New Minimum: -58.24866551303876 > -61.34744968735757
WOLF (strong): th(1.8144000000000004E7)=-61.34744968735757; dx=1.3128762271983108E-7 evalInputDelta=4.760659901030692
END: th(6048000.000000001)=-61.17099527598347; dx=-3.909516328835804E-7 evalInputDelta=4.584205489656597
Fitness changed from -56.586789786326875 to -61.34744968735757
Iteration 6 complete. Error: -61.34744968735757 Total: 32.9154; Orientation: 2.4878; Line Search: 23.7476
Adding measurement 154c8411 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.602e+04, gradient 1.085e-03, dot -0.365; [25b2f0ef-63b0-4491-af0b-2219757414d1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -61.34744968735757, -42.67452601783015, -38.07621083902471, -35.158202706832
LBFGS Accumulation History: 3 points
Removed measurement 154c8411 to history. Total: 3
th(0)=-61.34744968735757;dx=-1.174576326009951E-6
New Minimum: -61.34744968735757 > -69.72237550756017
WOLF (strong): th(1.3030021005312836E7)=-69.72237550756017; dx=6.851878267104887E-8 evalInputDelta=8.374925820202606
END: th(6515010.502656418)=-68.03735132011481; dx=-6.016631744974501E-7 evalInputDelta=6.689901632757241
Fitness changed from -61.34744968735757 to -69.72237550756017
Iteration 7 complete. Error: -69.72237550756017 Total: 26.3875; Orientation: 2.4996; Line Search: 17.7527
Adding measurement 66f1e0b1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.608e+04, gradient 8.496e-04, dot -0.482; [25b2f0ef-63b0-4491-af0b-2219757414d1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -69.72237550756017, -42.67452601783015, -38.07621083902471, -35.158202706832
LBFGS Accumulation History: 3 points
Removed measurement 66f1e0b1 to history. Total: 3
th(0)=-69.72237550756017;dx=-7.21803542830837E-7
New Minimum: -69.72237550756017 > -74.43732072988871
WOLF (strong): th(1.4036164632845048E7)=-74.43732072988871; dx=4.337692119834122E-8 evalInputDelta=4.714945222328538
END: th(7018082.316422524)=-73.71509726126934; dx=-2.93953027066095E-7 evalInputDelta=3.992721753709162
Fitness changed from -69.72237550756017 to -74.43732072988871
Iteration 8 complete. Error: -74.43732072988871 Total: 26.2128; Orientation: 2.4447; Line Search: 17.7629
Adding measurement 77d0f7b7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.581e+04, gradient 8.404e-04, dot -0.467; [25b2f0ef-63b0-4491-af0b-2219757414d1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -74.43732072988871, -42.67452601783015, -38.07621083902471, -35.158202706832
LBFGS Accumulation History: 3 points
Removed measurement 77d0f7b7 to history. Total: 3
th(0)=-74.43732072988871;dx=-7.05773670831808E-7
New Minimum: -74.43732072988871 > -78.64289778854722
WOLF (strong): th(1.5120000000000006E7)=-78.64289778854722; dx=1.1661787157678097E-7 evalInputDelta=4.2055770586585055
END: th(7560000.000000003)=-78.4756710885034; dx=-2.3779711539240192E-7 evalInputDelta=4.038350358614693
Fitness changed from -74.43732072988871 to -78.64289778854722
Iteration 9 complete. Error: -78.64289778854722 Total: 26.4995; Orientation: 2.4672; Line Search: 18.2206
Adding measurement 266c1a6f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.138e+04, gradient 8.562e-04, dot -0.432; [25b2f0ef-63b0-4491-af0b-2219757414d1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -78.64289778854722, -42.67452601783015, -38.07621083902471, -35.158202706832
LBFGS Accumulation History: 3 points
Removed measurement 266c1a6f to history. Total: 3
th(0)=-78.64289778854722;dx=-7.331103693824286E-7
New Minimum: -78.64289778854722 > -82.75637190802026
WOLF (strong): th(1.6287526256641047E7)=-82.75637190802026; dx=1.8603972430913094E-7 evalInputDelta=4.1134741194730395
New Minimum: -82.75637190802026 > -83.05522026873251
END: th(8143763.128320524)=-83.05522026873251; dx=-2.1372855741436807E-7 evalInputDelta=4.412322480185296
Fitness changed from -78.64289778854722 to -83.05522026873251
Iteration 10 complete. Error: -83.05522026873251 Total: 26.0046; Orientation: 2.4653; Line Search: 17.6144
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -83.05522026873251 (> -Infinity) after 294.738s (< 5400.000s)
Returns
-83.05522026873251