BasicOptimizer.scala:89 executed in 94.29 seconds (1.569 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 282939915665300
Reset training subject: 282941819098700
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -58.045309333801775 > -58.10908618878937
th(0)=-58.10908618878937;dx=-4.6196468838923345E-6
New Minimum: -58.10908618878937 > -58.10908941410205
END: th(2.154434690031884)=-58.10908941410205; dx=-3.4142847630100013E-6 evalInputDelta=3.225312681820469E-6
Fitness changed from -58.045309333801775 to -58.10908941410205
Iteration 1 complete. Error: -58.10908941410205 Total: 9.2656; Orientation: 0.0353; Line Search: 3.5872
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 6c079227 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-58.10908941410205;dx=-3.422954013724232E-6
New Minimum: -58.10908941410205 > -58.10910554066277
WOLFE (weak): th(4.641588833612779)=-58.10910554066277; dx=-3.422886203826082E-6 evalInputDelta=1.6126560716145377E-5
New Minimum: -58.10910554066277 > -58.10912166721901
WOLFE (weak): th(9.283177667225559)=-58.10912166721901; dx=-3.42288497220387E-6 evalInputDelta=3.225311695587152E-5
New Minimum: -58.10912166721901 > -58.109186173399216
WOLFE (weak): th(27.849533001676676)=-58.109186173399216; dx=-3.422874306068006E-6 evalInputDelta=9.675929716479459E-5
New Minimum: -58.109186173399216 > -58.10946999974143
WOLFE (weak): th(111.3981320067067)=-58.10946999974143; dx=-3.422869360577578E-6 evalInputDelta=3.805856393768181E-4
New Minimum: -58.10946999974143 > -58.110995542580184
WOLFE (weak): th(556.9906600335335)=-58.110995542580184; dx=-3.422899809196543E-6 evalInputDelta=0.0019061284781329846
New Minimum: -58.110995542580184 > -58.120528471944084
WOLFE (weak): th(3341.9439602012008)=-58.120528471944084; dx=-3.4220184503447117E-6 evalInputDelta=0.011439057842032696
New Minimum: -58.120528471944084 > -58.18907372411402
WOLFE (weak): th(23393.607721408407)=-58.18907372411402; dx=-3.414379647301809E-6 evalInputDelta=0.07998431001196593
New Minimum: -58.18907372411402 > -58.74223934769247
WOLFE (weak): th(187148.86177126726)=-58.74223934769247; dx=-3.3409491621421007E-6 evalInputDelta=0.6331499335904169
New Minimum: -58.74223934769247 > -63.268684036359154
END: th(1684339.7559414054)=-63.268684036359154; dx=-2.7074428680413985E-6 evalInputDelta=5.159594622257103
Fitness changed from -58.10908941410205 to -63.268684036359154
Iteration 2 complete. Error: -63.268684036359154 Total: 21.2618; Orientation: 0.0583; Line Search: 19.0483
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 1b716850 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-63.268684036359154;dx=-2.459624368017129E-6
New Minimum: -63.268684036359154 > -70.59188217214499
END: th(3628800.0000000005)=-70.59188217214499; dx=-1.574709892506254E-6 evalInputDelta=7.323198135785837
Fitness changed from -63.268684036359154 to -70.59188217214499
Iteration 3 complete. Error: -70.59188217214499 Total: 5.8995; Orientation: 0.0580; Line Search: 3.7095
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 1c8caae5 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-70.59188217214499;dx=-1.5238049172997439E-6
New Minimum: -70.59188217214499 > -79.4213717622533
END: th(7818012.603187701)=-79.4213717622533; dx=-7.782675310622392E-7 evalInputDelta=8.82948959010831
Fitness changed from -70.59188217214499 to -79.4213717622533
Iteration 4 complete. Error: -79.4213717622533 Total: 6.1138; Orientation: 0.0541; Line Search: 3.7423
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 7c250f53 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.403e+04, gradient 1.074e-03, dot -0.726; [a5bd5d6e-c98e-4ebc-ad1e-6d71d8c619ae = 1.000/1.000e+00]
Orientation rejected. Popping history element from -79.4213717622533, -70.59188217214499, -63.268684036359154, -58.10908941410205
LBFGS Accumulation History: 3 points
Removed measurement 7c250f53 to history. Total: 3
th(0)=-79.4213717622533;dx=-1.1545050565874574E-6
New Minimum: -79.4213717622533 > -87.59651855349973
END: th(1.6843397559414055E7)=-87.59651855349973; dx=-1.2047646586994015E-7 evalInputDelta=8.175146791246434
Fitness changed from -79.4213717622533 to -87.59651855349973
Iteration 5 complete. Error: -87.59651855349973 Total: 6.7352; Orientation: 1.0013; Line Search: 3.6222
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 235f6b01 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.207e+04, gradient 1.342e-03, dot -0.451; [a5bd5d6e-c98e-4ebc-ad1e-6d71d8c619ae = 1.000/1.000e+00]
Orientation rejected. Popping history element from -87.59651855349973, -70.59188217214499, -63.268684036359154, -58.10908941410205
LBFGS Accumulation History: 3 points
Removed measurement 235f6b01 to history. Total: 3
th(0)=-87.59651855349973;dx=-1.8013655628402042E-6
Armijo: th(3.628800000000001E7)=-79.81946645939531; dx=4.459549097155085E-7 evalInputDelta=-7.77705209410442
New Minimum: -87.59651855349973 > -89.88960752208565
WOLF (strong): th(1.8144000000000004E7)=-89.88960752208565; dx=6.431687168728147E-7 evalInputDelta=2.29308896858592
New Minimum: -89.88960752208565 > -94.4380065050858
END: th(6048000.000000001)=-94.4380065050858; dx=-3.80831120982499E-7 evalInputDelta=6.8414879515860605
Fitness changed from -87.59651855349973 to -94.4380065050858
Iteration 6 complete. Error: -94.4380065050858 Total: 10.3591; Orientation: 0.9629; Line Search: 7.3547
Adding measurement 48f27fc7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.591e+04, gradient 9.290e-04, dot -0.617; [a5bd5d6e-c98e-4ebc-ad1e-6d71d8c619ae = 1.000/1.000e+00]
Orientation rejected. Popping history element from -94.4380065050858, -70.59188217214499, -63.268684036359154, -58.10908941410205
LBFGS Accumulation History: 3 points
Removed measurement 48f27fc7 to history. Total: 3
th(0)=-94.4380065050858;dx=-8.63124336170591E-7
New Minimum: -94.4380065050858 > -99.1432154680449
END: th(1.3030021005312836E7)=-99.1432154680449; dx=-1.3771506632167582E-8 evalInputDelta=4.705208962959105
Fitness changed from -94.4380065050858 to -99.1432154680449
Iteration 7 complete. Error: -99.1432154680449 Total: 6.6046; Orientation: 0.9987; Line Search: 3.8271
Adding measurement e90380e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.712e+04, gradient 1.172e-03, dot -0.451; [a5bd5d6e-c98e-4ebc-ad1e-6d71d8c619ae = 1.000/1.000e+00]
Orientation rejected. Popping history element from -99.1432154680449, -70.59188217214499, -63.268684036359154, -58.10908941410205
LBFGS Accumulation History: 3 points
Removed measurement e90380e to history. Total: 3
th(0)=-99.1432154680449;dx=-1.3730320042219227E-6
Armijo: th(2.8072329265690096E7)=-94.3930211911049; dx=4.39212125113247E-7 evalInputDelta=-4.750194276939993
New Minimum: -99.1432154680449 > -101.5035773352177
WOLF (strong): th(1.4036164632845048E7)=-101.5035773352177; dx=5.315728782012631E-7 evalInputDelta=2.3603618671727986
New Minimum: -101.5035773352177 > -103.57235547179175
END: th(4678721.544281683)=-103.57235547179175; dx=-4.4831659404654363E-7 evalInputDelta=4.429140003746852
Fitness changed from -99.1432154680449 to -103.57235547179175
Iteration 8 complete. Error: -103.57235547179175 Total: 10.4311; Orientation: 0.9907; Line Search: 7.6220
Adding measurement 6c25e59b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.581e+04, gradient 7.497e-04, dot -0.677; [a5bd5d6e-c98e-4ebc-ad1e-6d71d8c619ae = 1.000/1.000e+00]
Orientation rejected. Popping history element from -103.57235547179175, -70.59188217214499, -63.268684036359154, -58.10908941410205
LBFGS Accumulation History: 3 points
Removed measurement 6c25e59b to history. Total: 3
th(0)=-103.57235547179175;dx=-5.62111040014854E-7
New Minimum: -103.57235547179175 > -107.38367323388309
END: th(1.0080000000000004E7)=-107.38367323388309; dx=-2.2829743517397767E-7 evalInputDelta=3.8113177620913348
Fitness changed from -103.57235547179175 to -107.38367323388309
Iteration 9 complete. Error: -107.38367323388309 Total: 6.6083; Orientation: 0.9311; Line Search: 3.7051
Adding measurement 69260129 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.717e+04, gradient 8.298e-04, dot -0.575; [a5bd5d6e-c98e-4ebc-ad1e-6d71d8c619ae = 1.000/1.000e+00]
Orientation rejected. Popping history element from -107.38367323388309, -70.59188217214499, -63.268684036359154, -58.10908941410205
LBFGS Accumulation History: 3 points
Removed measurement 69260129 to history. Total: 3
th(0)=-107.38367323388309;dx=-6.885662135262905E-7
New Minimum: -107.38367323388309 > -109.08722347152232
WOLF (strong): th(2.1716701675521396E7)=-109.08722347152232; dx=1.518906299594306E-7 evalInputDelta=1.7035502376392344
New Minimum: -109.08722347152232 > -110.22931043271339
WOLF (strong): th(1.0858350837760698E7)=-110.22931043271339; dx=2.345813344271663E-8 evalInputDelta=2.8456371988303033
END: th(3619450.279253566)=-109.30267716141576; dx=-3.635291233344969E-7 evalInputDelta=1.9190039275326711
Fitness changed from -107.38367323388309 to -110.22931043271339
Iteration 10 complete. Error: -110.22931043271339 Total: 10.6538; Orientation: 0.9499; Line Search: 7.8722
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -110.22931043271339 (> -Infinity) after 94.285s (< 5400.000s)
Returns
-110.22931043271339