BasicOptimizer.scala:89 executed in 128.15 seconds (1.707 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 73521896795600
Reset training subject: 73524599555300
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -18.550710771614874 > -18.59576571533214
th(0)=-18.59576571533214;dx=-9.507214309472095E-7
New Minimum: -18.59576571533214 > -18.5957671034824
WOLFE (weak): th(2.154434690031884)=-18.5957671034824; dx=-8.771244093857229E-7 evalInputDelta=1.388150263181842E-6
New Minimum: -18.5957671034824 > -18.595769879782615
WOLFE (weak): th(4.308869380063768)=-18.595769879782615; dx=-8.772901881018365E-7 evalInputDelta=4.1644504769067225E-6
New Minimum: -18.595769879782615 > -18.59577820868077
WOLFE (weak): th(12.926608140191302)=-18.59577820868077; dx=-8.77312054943207E-7 evalInputDelta=1.2493348631181789E-5
New Minimum: -18.59577820868077 > -18.595816382749568
WOLFE (weak): th(51.70643256076521)=-18.595816382749568; dx=-8.773019171228253E-7 evalInputDelta=5.066741742965064E-5
New Minimum: -18.595816382749568 > -18.59601905103878
WOLFE (weak): th(258.53216280382605)=-18.59601905103878; dx=-8.772416662383388E-7 evalInputDelta=2.533357066418773E-4
New Minimum: -18.59601905103878 > -18.5972863718256
WOLFE (weak): th(1551.1929768229563)=-18.5972863718256; dx=-8.770815665511056E-7 evalInputDelta=0.0015206564934615585
New Minimum: -18.5972863718256 > -18.606404926202842
WOLFE (weak): th(10858.350837760694)=-18.606404926202842; dx=-8.767316426662378E-7 evalInputDelta=0.010639210870703408
New Minimum: -18.606404926202842 > -18.68062406733012
WOLFE (weak): th(86866.80670208555)=-18.68062406733012; dx=-8.715027061145396E-7 evalInputDelta=0.08485835199797975
New Minimum: -18.68062406733012 > -19.326797954822958
END: th(781801.26031877)=-19.326797954822958; dx=-7.838207972480783E-7 evalInputDelta=0.7310322394908191
Fitness changed from -18.550710771614874 to -19.326797954822958
Iteration 1 complete. Error: -19.326797954822958 Total: 33.6093; Orientation: 0.0384; Line Search: 25.9181
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement d49a845 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-19.326797954822958;dx=-7.801302565372456E-7
New Minimum: -19.326797954822958 > -20.637516570347316
END: th(1684339.7559414052)=-20.637516570347316; dx=-6.005925452785147E-7 evalInputDelta=1.3107186155243582
Fitness changed from -19.326797954822958 to -20.637516570347316
Iteration 2 complete. Error: -20.637516570347316 Total: 8.2990; Orientation: 0.0590; Line Search: 5.2649
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 3aecc728 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-20.637516570347316;dx=-6.622916848886412E-7
New Minimum: -20.637516570347316 > -22.600953323085708
END: th(3628800.0)=-22.600953323085708; dx=-3.5428434200681025E-7 evalInputDelta=1.9634367527383922
Fitness changed from -20.637516570347316 to -22.600953323085708
Iteration 3 complete. Error: -22.600953323085708 Total: 7.9668; Orientation: 0.0529; Line Search: 4.9913
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 6372a114 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-22.600953323085708;dx=-6.669635033705996E-7
New Minimum: -22.600953323085708 > -24.57241895806237
END: th(7818012.6031877)=-24.57241895806237; dx=-1.2203716416264577E-7 evalInputDelta=1.9714656349766635
Fitness changed from -22.600953323085708 to -24.57241895806237
Iteration 4 complete. Error: -24.57241895806237 Total: 8.3676; Orientation: 0.0607; Line Search: 5.0796
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 2ef73762 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.929e+04, gradient 7.785e-04, dot -0.519; [c9b50a82-ccd3-4059-82be-9d2ff5d62220 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -24.57241895806237, -22.600953323085708, -20.637516570347316, -19.326797954822958
LBFGS Accumulation History: 3 points
Removed measurement 2ef73762 to history. Total: 3
th(0)=-24.57241895806237;dx=-6.056590020166382E-7
New Minimum: -24.57241895806237 > -26.80661658677779
END: th(1.684339755941405E7)=-26.80661658677779; dx=-2.104694818299773E-9 evalInputDelta=2.2341976287154175
Fitness changed from -24.57241895806237 to -26.80661658677779
Iteration 5 complete. Error: -26.80661658677779 Total: 9.3803; Orientation: 1.1177; Line Search: 5.3176
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 13a78b8d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.007e+04, gradient 7.005e-04, dot -0.346; [c9b50a82-ccd3-4059-82be-9d2ff5d62220 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -26.80661658677779, -22.600953323085708, -20.637516570347316, -19.326797954822958
LBFGS Accumulation History: 3 points
Removed measurement 13a78b8d to history. Total: 3
th(0)=-26.80661658677779;dx=-4.907373143444693E-7
New Minimum: -26.80661658677779 > -29.68008731283358
WOLF (strong): th(3.6288E7)=-29.68008731283358; dx=4.466494425509508E-8 evalInputDelta=2.8734707260557926
New Minimum: -29.68008731283358 > -30.587032905599372
WOLF (strong): th(1.8144E7)=-30.587032905599372; dx=1.8445956474854443E-8 evalInputDelta=3.780416318821583
END: th(6048000.0)=-29.529472893410915; dx=-2.704609979759037E-7 evalInputDelta=2.722856306633126
Fitness changed from -26.80661658677779 to -30.587032905599372
Iteration 6 complete. Error: -30.587032905599372 Total: 14.6761; Orientation: 1.0568; Line Search: 10.4443
Adding measurement 7164060a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.728e+04, gradient 6.980e-04, dot -0.310; [c9b50a82-ccd3-4059-82be-9d2ff5d62220 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -30.587032905599372, -22.600953323085708, -20.637516570347316, -19.326797954822958
LBFGS Accumulation History: 3 points
Removed measurement 7164060a to history. Total: 3
th(0)=-30.587032905599372;dx=-4.868940079838318E-7
New Minimum: -30.587032905599372 > -33.85471882418249
WOLF (strong): th(1.3030021005312834E7)=-33.85471882418249; dx=4.180463794304687E-8 evalInputDelta=3.267685918583119
END: th(6515010.502656417)=-33.3764635072651; dx=-2.2323523311402074E-7 evalInputDelta=2.789430601665728
Fitness changed from -30.587032905599372 to -33.85471882418249
Iteration 7 complete. Error: -33.85471882418249 Total: 11.1582; Orientation: 1.0226; Line Search: 7.6222
Adding measurement 2831fc6f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.377e+04, gradient 6.817e-04, dot -0.330; [c9b50a82-ccd3-4059-82be-9d2ff5d62220 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -33.85471882418249, -22.600953323085708, -20.637516570347316, -19.326797954822958
LBFGS Accumulation History: 3 points
Removed measurement 2831fc6f to history. Total: 3
th(0)=-33.85471882418249;dx=-4.6446166136460407E-7
New Minimum: -33.85471882418249 > -35.18116095611954
WOLF (strong): th(1.4036164632845046E7)=-35.18116095611954; dx=1.3635841307262778E-7 evalInputDelta=1.3264421319370499
New Minimum: -35.18116095611954 > -35.972282827183456
END: th(7018082.316422523)=-35.972282827183456; dx=-9.545862298788773E-9 evalInputDelta=2.1175640030009646
Fitness changed from -33.85471882418249 to -35.972282827183456
Iteration 8 complete. Error: -35.972282827183456 Total: 11.3644; Orientation: 0.9762; Line Search: 7.7218
Adding measurement 682815b6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.187e+04, gradient 6.258e-04, dot -0.389; [c9b50a82-ccd3-4059-82be-9d2ff5d62220 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -35.972282827183456, -22.600953323085708, -20.637516570347316, -19.326797954822958
LBFGS Accumulation History: 3 points
Removed measurement 682815b6 to history. Total: 3
th(0)=-35.972282827183456;dx=-3.915009134684282E-7
Armijo: th(1.5120000000000004E7)=-35.909736954757534; dx=7.314292355285308E-8 evalInputDelta=-0.0625458724259218
New Minimum: -35.972282827183456 > -36.689687208340594
WOLF (strong): th(7560000.000000002)=-36.689687208340594; dx=1.0005447043221609E-7 evalInputDelta=0.717404381157138
New Minimum: -36.689687208340594 > -36.82316609407842
END: th(2520000.0000000005)=-36.82316609407842; dx=-1.7309578909192463E-7 evalInputDelta=0.8508832668949609
Fitness changed from -35.972282827183456 to -36.82316609407842
Iteration 9 complete. Error: -36.82316609407842 Total: 14.1629; Orientation: 0.9769; Line Search: 10.3986
Adding measurement 62b04f2b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.515e+04, gradient 4.942e-04, dot -0.498; [c9b50a82-ccd3-4059-82be-9d2ff5d62220 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -36.82316609407842, -22.600953323085708, -20.637516570347316, -19.326797954822958
LBFGS Accumulation History: 3 points
Removed measurement 62b04f2b to history. Total: 3
th(0)=-36.82316609407842;dx=-2.4403917214708046E-7
New Minimum: -36.82316609407842 > -37.45488617976044
END: th(5429175.418880348)=-37.45488617976044; dx=-2.5599794630133047E-8 evalInputDelta=0.6317200856820264
Fitness changed from -36.82316609407842 to -37.45488617976044
Iteration 10 complete. Error: -37.45488617976044 Total: 8.6472; Orientation: 1.0013; Line Search: 5.1448
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -37.45488617976044 (> -Infinity) after 128.149s (< 5400.000s)
Returns
-37.45488617976044