BasicOptimizer.scala:89 executed in 307.95 seconds (4.347 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 660606547462600
Reset training subject: 660612738008800
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -34.95607109954808 > -35.02952351866833
th(0)=-35.02952351866833;dx=-1.8616713985180118E-6
New Minimum: -35.02952351866833 > -35.02952675282222
WOLFE (weak): th(2.154434690031884)=-35.02952675282222; dx=-1.695919504330087E-6 evalInputDelta=3.2341538940272585E-6
New Minimum: -35.02952675282222 > -35.029529986975824
WOLFE (weak): th(4.308869380063768)=-35.029529986975824; dx=-1.6961690253733744E-6 evalInputDelta=6.468307496731995E-6
New Minimum: -35.029529986975824 > -35.029549391891145
WOLFE (weak): th(12.926608140191302)=-35.029549391891145; dx=-1.696031428022626E-6 evalInputDelta=2.5873222817551778E-5
New Minimum: -35.029549391891145 > -35.02962162120372
WOLFE (weak): th(51.70643256076521)=-35.02962162120372; dx=-1.6959489753250769E-6 evalInputDelta=9.81025353894438E-5
New Minimum: -35.02962162120372 > -35.03001187252531
WOLFE (weak): th(258.53216280382605)=-35.03001187252531; dx=-1.6959106455822014E-6 evalInputDelta=4.883538569799839E-4
New Minimum: -35.03001187252531 > -35.0324578515396
WOLFE (weak): th(1551.1929768229563)=-35.0324578515396; dx=-1.6955898912034471E-6 evalInputDelta=0.0029343328712698735
New Minimum: -35.0324578515396 > -35.05005976738006
WOLFE (weak): th(10858.350837760694)=-35.05005976738006; dx=-1.6946792974310168E-6 evalInputDelta=0.02053624871173554
New Minimum: -35.05005976738006 > -35.19345329509453
WOLFE (weak): th(86866.80670208555)=-35.19345329509453; dx=-1.6872183074755748E-6 evalInputDelta=0.1639297764262011
New Minimum: -35.19345329509453 > -36.46904262079307
END: th(781801.26031877)=-36.46904262079307; dx=-1.6014371816488947E-6 evalInputDelta=1.4395191021247413
Fitness changed from -34.95607109954808 to -36.46904262079307
Iteration 1 complete. Error: -36.46904262079307 Total: 80.3920; Orientation: 0.0947; Line Search: 62.1882
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 3e60cf6f to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-36.46904262079307;dx=-1.5531572950946204E-6
New Minimum: -36.46904262079307 > -39.19727129861348
END: th(1684339.7559414052)=-39.19727129861348; dx=-1.352259277219089E-6 evalInputDelta=2.728228677820411
Fitness changed from -36.46904262079307 to -39.19727129861348
Iteration 2 complete. Error: -39.19727129861348 Total: 19.8720; Orientation: 0.1308; Line Search: 12.1343
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 726b2cbf to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-39.19727129861348;dx=-1.263111805442591E-6
New Minimum: -39.19727129861348 > -43.640788291041176
END: th(3628800.0)=-43.640788291041176; dx=-9.422637366687141E-7 evalInputDelta=4.443516992427696
Fitness changed from -39.19727129861348 to -43.640788291041176
Iteration 3 complete. Error: -43.640788291041176 Total: 19.0105; Orientation: 0.1341; Line Search: 11.9018
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 22ed5c8e to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-43.640788291041176;dx=-9.095839247497819E-7
New Minimum: -43.640788291041176 > -50.01698637365137
END: th(7818012.6031877)=-50.01698637365137; dx=-5.846785022332437E-7 evalInputDelta=6.376198082610195
Fitness changed from -43.640788291041176 to -50.01698637365137
Iteration 4 complete. Error: -50.01698637365137 Total: 19.8925; Orientation: 0.1480; Line Search: 12.5572
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 3336b41 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.454e+04, gradient 8.480e-04, dot -0.797; [52e274a4-6b66-4809-868b-7069711b45eb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -50.01698637365137, -43.640788291041176, -39.19727129861348, -36.46904262079307
LBFGS Accumulation History: 3 points
Removed measurement 3336b41 to history. Total: 3
th(0)=-50.01698637365137;dx=-7.191248332464328E-7
New Minimum: -50.01698637365137 > -57.99760943665205
END: th(1.684339755941405E7)=-57.99760943665205; dx=-2.611688279036363E-7 evalInputDelta=7.980623063000678
Fitness changed from -50.01698637365137 to -57.99760943665205
Iteration 5 complete. Error: -57.99760943665205 Total: 22.7107; Orientation: 2.6674; Line Search: 12.5943
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 59c3915d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.609e+04, gradient 9.390e-04, dot -0.542; [52e274a4-6b66-4809-868b-7069711b45eb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -57.99760943665205, -43.640788291041176, -39.19727129861348, -36.46904262079307
LBFGS Accumulation History: 3 points
Removed measurement 59c3915d to history. Total: 3
th(0)=-57.99760943665205;dx=-8.814875210288001E-7
New Minimum: -57.99760943665205 > -60.897418123591336
WOLF (strong): th(3.6288E7)=-60.897418123591336; dx=7.327152764923246E-8 evalInputDelta=2.899808686939288
New Minimum: -60.897418123591336 > -63.00382097638774
WOLF (strong): th(1.8144E7)=-63.00382097638774; dx=7.335456788505501E-8 evalInputDelta=5.006211539735695
END: th(6048000.0)=-62.29369887045724; dx=-3.817463987891508E-7 evalInputDelta=4.296089433805193
Fitness changed from -57.99760943665205 to -63.00382097638774
Iteration 6 complete. Error: -63.00382097638774 Total: 34.9911; Orientation: 2.6581; Line Search: 24.8675
Adding measurement 28f765de to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.039e+04, gradient 1.045e-03, dot -0.408; [52e274a4-6b66-4809-868b-7069711b45eb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -63.00382097638774, -43.640788291041176, -39.19727129861348, -36.46904262079307
LBFGS Accumulation History: 3 points
Removed measurement 28f765de to history. Total: 3
th(0)=-63.00382097638774;dx=-1.0925448965037046E-6
New Minimum: -63.00382097638774 > -70.92885491422885
WOLF (strong): th(1.3030021005312834E7)=-70.92885491422885; dx=4.0860804987139034E-8 evalInputDelta=7.925033937841107
END: th(6515010.502656417)=-69.24361556101071; dx=-5.720853516314594E-7 evalInputDelta=6.23979458462297
Fitness changed from -63.00382097638774 to -70.92885491422885
Iteration 7 complete. Error: -70.92885491422885 Total: 27.4916; Orientation: 2.5452; Line Search: 18.9464
Adding measurement 4e54bc6e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.893e+04, gradient 8.347e-04, dot -0.510; [52e274a4-6b66-4809-868b-7069711b45eb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -70.92885491422885, -43.640788291041176, -39.19727129861348, -36.46904262079307
LBFGS Accumulation History: 3 points
Removed measurement 4e54bc6e to history. Total: 3
th(0)=-70.92885491422885;dx=-6.966049376580955E-7
New Minimum: -70.92885491422885 > -75.55909037320137
WOLF (strong): th(1.4036164632845046E7)=-75.55909037320137; dx=3.17885860053542E-8 evalInputDelta=4.630235458972521
END: th(7018082.316422523)=-74.80258862851308; dx=-2.887721243240346E-7 evalInputDelta=3.873733714284228
Fitness changed from -70.92885491422885 to -75.55909037320137
Iteration 8 complete. Error: -75.55909037320137 Total: 27.0899; Orientation: 2.5663; Line Search: 18.2115
Adding measurement 13b02484 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.718e+04, gradient 8.280e-04, dot -0.489; [52e274a4-6b66-4809-868b-7069711b45eb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -75.55909037320137, -43.640788291041176, -39.19727129861348, -36.46904262079307
LBFGS Accumulation History: 3 points
Removed measurement 13b02484 to history. Total: 3
th(0)=-75.55909037320137;dx=-6.849582203558583E-7
New Minimum: -75.55909037320137 > -79.71175417003072
WOLF (strong): th(1.5120000000000004E7)=-79.71175417003072; dx=1.0361394855502926E-7 evalInputDelta=4.152663796829344
END: th(7560000.000000002)=-79.4921270399969; dx=-2.3270352957067503E-7 evalInputDelta=3.9330366667955303
Fitness changed from -75.55909037320137 to -79.71175417003072
Iteration 9 complete. Error: -79.71175417003072 Total: 26.7034; Orientation: 2.5333; Line Search: 18.4181
Adding measurement 5d5fda5a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.202e+04, gradient 8.397e-04, dot -0.453; [52e274a4-6b66-4809-868b-7069711b45eb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -79.71175417003072, -43.640788291041176, -39.19727129861348, -36.46904262079307
LBFGS Accumulation History: 3 points
Removed measurement 5d5fda5a to history. Total: 3
th(0)=-79.71175417003072;dx=-7.044365217670239E-7
New Minimum: -79.71175417003072 > -83.85644278635688
WOLF (strong): th(1.6287526256641045E7)=-83.85644278635688; dx=1.648987522737325E-7 evalInputDelta=4.144688616326164
New Minimum: -83.85644278635688 > -84.01945749943228
END: th(8143763.128320523)=-84.01945749943228; dx=-2.1708723422709805E-7 evalInputDelta=4.307703329401562
Fitness changed from -79.71175417003072 to -84.01945749943228
Iteration 10 complete. Error: -84.01945749943228 Total: 28.5920; Orientation: 2.5802; Line Search: 19.9345
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -84.01945749943228 (> -Infinity) after 307.954s (< 5400.000s)
Returns
-84.01945749943228