BasicOptimizer.scala:89 executed in 149.14 seconds (2.569 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 461298042895600
Reset training subject: 461300873083800
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -100.22438805376507 > -100.2285903744025
th(0)=-100.2285903744025;dx=-1.5245039533883304E-5
New Minimum: -100.2285903744025 > -100.22861809361068
END: th(2.154434690031884)=-100.22861809361068; dx=-1.242028679438933E-5 evalInputDelta=2.7719208176790744E-5
Fitness changed from -100.22438805376507 to -100.22861809361068
Iteration 1 complete. Error: -100.22861809361068 Total: 14.2677; Orientation: 0.0657; Line Search: 5.6518
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 3246c821 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-100.22861809361068;dx=-1.2461826708272997E-5
New Minimum: -100.22861809361068 > -100.22868461967903
WOLFE (weak): th(4.641588833612779)=-100.22868461967903; dx=-1.2443461549383744E-5 evalInputDelta=6.65260683518909E-5
New Minimum: -100.22868461967903 > -100.22874560186958
WOLFE (weak): th(9.283177667225559)=-100.22874560186958; dx=-1.2435578130980154E-5 evalInputDelta=1.2750825889895623E-4
New Minimum: -100.22874560186958 > -100.22898398644021
WOLFE (weak): th(27.849533001676676)=-100.22898398644021; dx=-1.242489361297883E-5 evalInputDelta=3.6589282953514157E-4
New Minimum: -100.22898398644021 > -100.23007611315451
WOLFE (weak): th(111.3981320067067)=-100.23007611315451; dx=-1.2395740693756813E-5 evalInputDelta=0.0014580195438327337
New Minimum: -100.23007611315451 > -100.23589134893368
WOLFE (weak): th(556.9906600335335)=-100.23589134893368; dx=-1.2408571182966658E-5 evalInputDelta=0.0072732553229997166
New Minimum: -100.23589134893368 > -100.27222754634026
WOLFE (weak): th(3341.9439602012008)=-100.27222754634026; dx=-1.2395261154951077E-5 evalInputDelta=0.04360945272958361
New Minimum: -100.27222754634026 > -100.53328250780639
WOLFE (weak): th(23393.607721408407)=-100.53328250780639; dx=-1.2359724729251798E-5 evalInputDelta=0.3046644141957131
New Minimum: -100.53328250780639 > -102.62254338202318
WOLFE (weak): th(187148.86177126726)=-102.62254338202318; dx=-1.1876560423506209E-5 evalInputDelta=2.393925288412504
New Minimum: -102.62254338202318 > -117.98405004306107
END: th(1684339.7559414054)=-117.98405004306107; dx=-7.682032304578089E-6 evalInputDelta=17.755431949450397
Fitness changed from -100.22861809361068 to -117.98405004306107
Iteration 2 complete. Error: -117.98405004306107 Total: 32.2714; Orientation: 0.1089; Line Search: 28.5473
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 99a4910 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-117.98405004306107;dx=-7.890500997631045E-6
New Minimum: -117.98405004306107 > -137.7553923165233
END: th(3628800.0000000005)=-137.7553923165233; dx=-3.3374995589624447E-6 evalInputDelta=19.77134227346224
Fitness changed from -117.98405004306107 to -137.7553923165233
Iteration 3 complete. Error: -137.7553923165233 Total: 9.3883; Orientation: 0.1038; Line Search: 5.6497
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 493e9cae to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-137.7553923165233;dx=-5.111028103878251E-6
New Minimum: -137.7553923165233 > -155.47992039193232
END: th(7818012.603187701)=-155.47992039193232; dx=-6.540748684184494E-7 evalInputDelta=17.72452807540901
Fitness changed from -137.7553923165233 to -155.47992039193232
Iteration 4 complete. Error: -155.47992039193232 Total: 9.2017; Orientation: 0.1032; Line Search: 5.7782
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 22e4876f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.740e+04, gradient 2.524e-03, dot -0.479; [d8fb1736-52e5-4d69-99a9-624b6eac25a4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -155.47992039193232, -137.7553923165233, -117.98405004306107, -100.22861809361068
LBFGS Accumulation History: 3 points
Removed measurement 22e4876f to history. Total: 3
th(0)=-155.47992039193232;dx=-6.372396352344809E-6
Armijo: th(1.6843397559414055E7)=-148.3365159920008; dx=1.8029724808574884E-6 evalInputDelta=-7.143404399931512
New Minimum: -155.47992039193232 > -166.10009656325624
WOLF (strong): th(8421698.779707028)=-166.10009656325624; dx=1.4118799731113934E-6 evalInputDelta=10.62017617132392
New Minimum: -166.10009656325624 > -167.8762443098485
END: th(2807232.926569009)=-167.8762443098485; dx=-2.2380749873447003E-6 evalInputDelta=12.396323917916163
Fitness changed from -155.47992039193232 to -167.8762443098485
Iteration 5 complete. Error: -167.8762443098485 Total: 16.7473; Orientation: 1.8646; Line Search: 11.6622
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement af6da7e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.882e+04, gradient 1.770e-03, dot -0.646; [d8fb1736-52e5-4d69-99a9-624b6eac25a4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -167.8762443098485, -137.7553923165233, -117.98405004306107, -100.22861809361068
LBFGS Accumulation History: 3 points
Removed measurement af6da7e to history. Total: 3
th(0)=-167.8762443098485;dx=-3.1185555363477093E-6
New Minimum: -167.8762443098485 > -178.96332869099595
END: th(6048000.000000001)=-178.96332869099595; dx=-9.341033037695906E-7 evalInputDelta=11.087084381147463
Fitness changed from -167.8762443098485 to -178.96332869099595
Iteration 6 complete. Error: -178.96332869099595 Total: 10.8749; Orientation: 1.9193; Line Search: 5.5805
Adding measurement 5faa4972 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.719e+04, gradient 1.941e-03, dot -0.558; [d8fb1736-52e5-4d69-99a9-624b6eac25a4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -178.96332869099595, -137.7553923165233, -117.98405004306107, -100.22861809361068
LBFGS Accumulation History: 3 points
Removed measurement 5faa4972 to history. Total: 3
th(0)=-178.96332869099595;dx=-3.756692359829544E-6
New Minimum: -178.96332869099595 > -180.91426094875166
WOLF (strong): th(1.3030021005312836E7)=-180.91426094875166; dx=9.639808440019744E-7 evalInputDelta=1.95093225775571
New Minimum: -180.91426094875166 > -186.91422225898864
WOLF (strong): th(6515010.502656418)=-186.91422225898864; dx=2.5537832749418207E-7 evalInputDelta=7.9508935679926935
END: th(2171670.167552139)=-185.065276599168; dx=-1.769182872098607E-6 evalInputDelta=6.101947908172036
Fitness changed from -178.96332869099595 to -186.91422225898864
Iteration 7 complete. Error: -186.91422225898864 Total: 16.0450; Orientation: 1.7268; Line Search: 11.3461
Adding measurement 3afdc604 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.011e+04, gradient 2.165e-03, dot -0.464; [d8fb1736-52e5-4d69-99a9-624b6eac25a4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -186.91422225898864, -137.7553923165233, -117.98405004306107, -100.22861809361068
LBFGS Accumulation History: 3 points
Removed measurement 3afdc604 to history. Total: 3
th(0)=-186.91422225898864;dx=-4.686650313682988E-6
New Minimum: -186.91422225898864 > -197.57094545029022
WOLF (strong): th(4678721.544281682)=-197.57094545029022; dx=4.211881404523315E-8 evalInputDelta=10.65672319130158
END: th(2339360.772140841)=-195.33740111202033; dx=-2.2096400004442817E-6 evalInputDelta=8.423178853031686
Fitness changed from -186.91422225898864 to -197.57094545029022
Iteration 8 complete. Error: -197.57094545029022 Total: 13.0489; Orientation: 1.7673; Line Search: 8.5995
Adding measurement 30ec341f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.985e+04, gradient 1.752e-03, dot -0.547; [d8fb1736-52e5-4d69-99a9-624b6eac25a4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -197.57094545029022, -137.7553923165233, -117.98405004306107, -100.22861809361068
LBFGS Accumulation History: 3 points
Removed measurement 30ec341f to history. Total: 3
th(0)=-197.57094545029022;dx=-3.0584563495799495E-6
New Minimum: -197.57094545029022 > -204.79808644448002
END: th(5040000.000000001)=-204.79808644448002; dx=-1.4289718459619191E-7 evalInputDelta=7.227140994189796
Fitness changed from -197.57094545029022 to -204.79808644448002
Iteration 9 complete. Error: -204.79808644448002 Total: 10.4264; Orientation: 1.7778; Line Search: 5.8766
Adding measurement 5d6d97c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.570e+04, gradient 1.717e-03, dot -0.537; [d8fb1736-52e5-4d69-99a9-624b6eac25a4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -204.79808644448002, -137.7553923165233, -117.98405004306107, -100.22861809361068
LBFGS Accumulation History: 3 points
Removed measurement 5d6d97c to history. Total: 3
th(0)=-204.79808644448002;dx=-2.939378021439799E-6
New Minimum: -204.79808644448002 > -206.53369255615357
WOLF (strong): th(1.0858350837760696E7)=-206.53369255615357; dx=1.0234097581079438E-6 evalInputDelta=1.7356061116735475
New Minimum: -206.53369255615357 > -211.30246679390945
WOLF (strong): th(5429175.418880348)=-211.30246679390945; dx=1.4048973500432887E-7 evalInputDelta=6.504380349429425
END: th(1809725.1396267826)=-209.112133951946; dx=-1.706761393873621E-6 evalInputDelta=4.314047507465972
Fitness changed from -204.79808644448002 to -211.30246679390945
Iteration 10 complete. Error: -211.30246679390945 Total: 16.2670; Orientation: 1.8236; Line Search: 11.4493
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -211.30246679390945 (> -Infinity) after 149.136s (< 5400.000s)
Returns
-211.30246679390945