BasicOptimizer.scala:89 executed in 132.97 seconds (1.654 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 73771668196500
Reset training subject: 73774282541600
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -25.290658671890018 > -25.365460999659902
th(0)=-25.365460999659902;dx=-7.616309857763394E-7
New Minimum: -25.365460999659902 > -25.36546303500371
WOLFE (weak): th(2.154434690031884)=-25.36546303500371; dx=-6.92831480633694E-7 evalInputDelta=2.0353438081599506E-6
New Minimum: -25.36546303500371 > -25.36546405267555
WOLFE (weak): th(4.308869380063768)=-25.36546405267555; dx=-6.928372772687955E-7 evalInputDelta=3.053015646514723E-6
New Minimum: -25.36546405267555 > -25.36547219404882
WOLFE (weak): th(12.926608140191302)=-25.36547219404882; dx=-6.926845843662545E-7 evalInputDelta=1.1194388918056575E-5
New Minimum: -25.36547219404882 > -25.3655027241753
WOLFE (weak): th(51.70643256076521)=-25.3655027241753; dx=-6.927485261926684E-7 evalInputDelta=4.1724515398300355E-5
New Minimum: -25.3655027241753 > -25.365663515568382
WOLFE (weak): th(258.53216280382605)=-25.365663515568382; dx=-6.927328634606598E-7 evalInputDelta=2.0251590848019418E-4
New Minimum: -25.365663515568382 > -25.366675053235653
WOLFE (weak): th(1551.1929768229563)=-25.366675053235653; dx=-6.926934302261991E-7 evalInputDelta=0.0012140535757509952
New Minimum: -25.366675053235653 > -25.373955102845578
WOLFE (weak): th(10858.350837760694)=-25.373955102845578; dx=-6.92269167083299E-7 evalInputDelta=0.008494103185675783
New Minimum: -25.373955102845578 > -25.433193315022496
WOLFE (weak): th(86866.80670208555)=-25.433193315022496; dx=-6.875837826372612E-7 evalInputDelta=0.06773231536259416
New Minimum: -25.433193315022496 > -25.9480325724457
END: th(781801.26031877)=-25.9480325724457; dx=-6.190056510569519E-7 evalInputDelta=0.582571572785799
Fitness changed from -25.290658671890018 to -25.9480325724457
Iteration 1 complete. Error: -25.9480325724457 Total: 33.6340; Orientation: 0.0363; Line Search: 25.8593
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 7e83ebce to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-25.9480325724457;dx=-6.053167065892664E-7
New Minimum: -25.9480325724457 > -26.98012178292885
END: th(1684339.7559414052)=-26.98012178292885; dx=-4.7292911319415186E-7 evalInputDelta=1.03208921048315
Fitness changed from -25.9480325724457 to -26.98012178292885
Iteration 2 complete. Error: -26.98012178292885 Total: 8.0251; Orientation: 0.0518; Line Search: 5.0840
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 1fdee900 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-26.98012178292885;dx=-5.069239600054359E-7
New Minimum: -26.98012178292885 > -28.56822435010865
END: th(3628800.0)=-28.56822435010865; dx=-3.031057650373717E-7 evalInputDelta=1.5881025671797993
Fitness changed from -26.98012178292885 to -28.56822435010865
Iteration 3 complete. Error: -28.56822435010865 Total: 8.3596; Orientation: 0.0592; Line Search: 5.2040
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 2cef6a99 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-28.56822435010865;dx=-5.018054190881057E-7
New Minimum: -28.56822435010865 > -30.295514606384145
END: th(7818012.6031877)=-30.295514606384145; dx=-1.1407620665863204E-7 evalInputDelta=1.727290256275495
Fitness changed from -28.56822435010865 to -30.295514606384145
Iteration 4 complete. Error: -30.295514606384145 Total: 8.4027; Orientation: 0.0585; Line Search: 5.3876
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 16282142 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.942e+04, gradient 7.369e-04, dot -0.548; [4b04de5e-790e-4043-b6ec-7a056b840622 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -30.295514606384145, -28.56822435010865, -26.98012178292885, -25.9480325724457
LBFGS Accumulation History: 3 points
Removed measurement 16282142 to history. Total: 3
th(0)=-30.295514606384145;dx=-5.429424113451698E-7
New Minimum: -30.295514606384145 > -31.75885671289035
WOLF (strong): th(1.684339755941405E7)=-31.75885671289035; dx=7.400056049231234E-9 evalInputDelta=1.4633421065062038
New Minimum: -31.75885671289035 > -32.00634054098476
WOLF (strong): th(8421698.779707026)=-32.00634054098476; dx=2.2060070608011095E-8 evalInputDelta=1.7108259346006136
END: th(2807232.9265690087)=-31.662494063840516; dx=-2.657742530891082E-7 evalInputDelta=1.3669794574563703
Fitness changed from -30.295514606384145 to -32.00634054098476
Iteration 5 complete. Error: -32.00634054098476 Total: 14.4423; Orientation: 1.0943; Line Search: 10.3265
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 1e3bfcdf to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.587e+04, gradient 7.724e-04, dot -0.432; [4b04de5e-790e-4043-b6ec-7a056b840622 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -32.00634054098476, -28.56822435010865, -26.98012178292885, -25.9480325724457
LBFGS Accumulation History: 3 points
Removed measurement 1e3bfcdf to history. Total: 3
th(0)=-32.00634054098476;dx=-5.961489351630875E-7
New Minimum: -32.00634054098476 > -34.18068720562751
WOLF (strong): th(6048000.0)=-34.18068720562751; dx=2.6898061009577034E-8 evalInputDelta=2.174346664642748
END: th(3024000.0)=-33.72817973736921; dx=-3.39553366269326E-7 evalInputDelta=1.721839196384451
Fitness changed from -32.00634054098476 to -34.18068720562751
Iteration 6 complete. Error: -34.18068720562751 Total: 11.7619; Orientation: 1.0111; Line Search: 7.7296
Adding measurement 6e43c292 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.605e+04, gradient 7.639e-04, dot -0.455; [4b04de5e-790e-4043-b6ec-7a056b840622 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -34.18068720562751, -28.56822435010865, -26.98012178292885, -25.9480325724457
LBFGS Accumulation History: 3 points
Removed measurement 6e43c292 to history. Total: 3
th(0)=-34.18068720562751;dx=-5.8270079323347E-7
New Minimum: -34.18068720562751 > -34.99133295098055
WOLF (strong): th(6515010.502656417)=-34.99133295098055; dx=1.7265366428669366E-7 evalInputDelta=0.8106457453530425
New Minimum: -34.99133295098055 > -35.4596724857483
END: th(3257505.2513282085)=-35.4596724857483; dx=-2.832104509714618E-8 evalInputDelta=1.2789852801207928
Fitness changed from -34.18068720562751 to -35.4596724857483
Iteration 7 complete. Error: -35.4596724857483 Total: 11.2657; Orientation: 1.0406; Line Search: 7.7487
Adding measurement 19ac7162 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.530e+04, gradient 6.633e-04, dot -0.550; [4b04de5e-790e-4043-b6ec-7a056b840622 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -35.4596724857483, -28.56822435010865, -26.98012178292885, -25.9480325724457
LBFGS Accumulation History: 3 points
Removed measurement 19ac7162 to history. Total: 3
th(0)=-35.4596724857483;dx=-4.3970740737420637E-7
New Minimum: -35.4596724857483 > -35.71976425365132
WOLF (strong): th(7018082.316422523)=-35.71976425365132; dx=5.8293437864347057E-8 evalInputDelta=0.26009176790302035
New Minimum: -35.71976425365132 > -36.022529981128606
WOLF (strong): th(3509041.1582112615)=-36.022529981128606; dx=6.422594909045194E-8 evalInputDelta=0.5628574953803067
END: th(1169680.3860704205)=-35.93155461593148; dx=-2.469875538118937E-7 evalInputDelta=0.47188213018318237
Fitness changed from -35.4596724857483 to -36.022529981128606
Iteration 8 complete. Error: -36.022529981128606 Total: 13.8338; Orientation: 1.0157; Line Search: 10.3420
Adding measurement 1f751201 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.547e+04, gradient 7.150e-04, dot -0.493; [4b04de5e-790e-4043-b6ec-7a056b840622 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -36.022529981128606, -28.56822435010865, -26.98012178292885, -25.9480325724457
LBFGS Accumulation History: 3 points
Removed measurement 1f751201 to history. Total: 3
th(0)=-36.022529981128606;dx=-5.110039374274085E-7
New Minimum: -36.022529981128606 > -36.82597146296439
WOLF (strong): th(2520000.0000000005)=-36.82597146296439; dx=4.1879554520227213E-8 evalInputDelta=0.8034414818357831
END: th(1260000.0000000002)=-36.645648330432756; dx=-3.237831787508451E-7 evalInputDelta=0.6231183493041499
Fitness changed from -36.022529981128606 to -36.82597146296439
Iteration 9 complete. Error: -36.82597146296439 Total: 11.4472; Orientation: 1.0127; Line Search: 7.8569
Adding measurement 28bf79e0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.100e+04, gradient 6.363e-04, dot -0.559; [4b04de5e-790e-4043-b6ec-7a056b840622 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -36.82597146296439, -28.56822435010865, -26.98012178292885, -25.9480325724457
LBFGS Accumulation History: 3 points
Removed measurement 28bf79e0 to history. Total: 3
th(0)=-36.82597146296439;dx=-4.0457734054384457E-7
New Minimum: -36.82597146296439 > -37.204513922792856
WOLF (strong): th(2714587.709440174)=-37.204513922792856; dx=7.742326985124628E-8 evalInputDelta=0.37854245982846635
New Minimum: -37.204513922792856 > -37.23088965319701
END: th(1357293.854720087)=-37.23088965319701; dx=-9.683419604123002E-8 evalInputDelta=0.4049181902326211
Fitness changed from -36.82597146296439 to -37.23088965319701
Iteration 10 complete. Error: -37.23088965319701 Total: 11.3184; Orientation: 0.9685; Line Search: 7.7408
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -37.23088965319701 (> -Infinity) after 132.970s (< 5400.000s)
Returns
-37.23088965319701