BasicOptimizer.scala:89 executed in 336.22 seconds (4.281 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 653261969184900
Reset training subject: 653268559214700
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -37.692004626586105 > -37.772173574784574
th(0)=-37.772173574784574;dx=-1.992984638671611E-6
New Minimum: -37.772173574784574 > -37.77217557433188
WOLFE (weak): th(2.154434690031884)=-37.77217557433188; dx=-1.909885448188337E-6 evalInputDelta=1.9995473081735327E-6
New Minimum: -37.77217557433188 > -37.77218157297319
WOLFE (weak): th(4.308869380063768)=-37.77218157297319; dx=-1.9096796908838496E-6 evalInputDelta=7.99818861452195E-6
New Minimum: -37.77218157297319 > -37.77219956889138
WOLFE (weak): th(12.926608140191302)=-37.77219956889138; dx=-1.9099085824643055E-6 evalInputDelta=2.5994106806592754E-5
New Minimum: -37.77219956889138 > -37.77228155018799
WOLFE (weak): th(51.70643256076521)=-37.77228155018799; dx=-1.9096110240186585E-6 evalInputDelta=1.0797540341656031E-4
New Minimum: -37.77228155018799 > -37.7727174477525
WOLFE (weak): th(258.53216280382605)=-37.7727174477525; dx=-1.909438239077354E-6 evalInputDelta=5.438729679241305E-4
New Minimum: -37.7727174477525 > -37.775436695135724
WOLFE (weak): th(1551.1929768229563)=-37.775436695135724; dx=-1.9097383319851467E-6 evalInputDelta=0.0032631203511499507
New Minimum: -37.775436695135724 > -37.795009500931776
WOLFE (weak): th(10858.350837760694)=-37.795009500931776; dx=-1.908161154967527E-6 evalInputDelta=0.0228359261472022
New Minimum: -37.795009500931776 > -37.95450015627413
WOLFE (weak): th(86866.80670208555)=-37.95450015627413; dx=-1.901427361967482E-6 evalInputDelta=0.18232658148955494
New Minimum: -37.95450015627413 > -39.37916584978225
WOLFE (weak): th(781801.26031877)=-39.37916584978225; dx=-1.8193642569754531E-6 evalInputDelta=1.6069922749976726
New Minimum: -39.37916584978225 > -49.384903693468345
END: th(7818012.6031877)=-49.384903693468345; dx=-7.815268132469853E-7 evalInputDelta=11.612730118683771
Fitness changed from -37.692004626586105 to -49.384903693468345
Iteration 1 complete. Error: -49.384903693468345 Total: 89.5337; Orientation: 0.0957; Line Search: 70.1267
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 2ce3f44e to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-49.384903693468345;dx=-1.015350136461732E-6
New Minimum: -49.384903693468345 > -60.00329570711195
END: th(1.684339755941405E7)=-60.00329570711195; dx=-2.815548795696702E-7 evalInputDelta=10.618392013643607
Fitness changed from -49.384903693468345 to -60.00329570711195
Iteration 2 complete. Error: -60.00329570711195 Total: 20.5710; Orientation: 0.1425; Line Search: 13.1286
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 4ec4bd24 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-60.00329570711195;dx=-9.487493029077923E-7
New Minimum: -60.00329570711195 > -65.31061226353204
WOLF (strong): th(3.6288E7)=-65.31061226353204; dx=1.0327138796192653E-7 evalInputDelta=5.3073165564200835
New Minimum: -65.31061226353204 > -67.35107011155344
WOLF (strong): th(1.8144E7)=-67.35107011155344; dx=1.3155128556617344E-8 evalInputDelta=7.347774404441488
END: th(6048000.0)=-65.01315755834736; dx=-5.323975529027124E-7 evalInputDelta=5.0098618512354065
Fitness changed from -60.00329570711195 to -67.35107011155344
Iteration 3 complete. Error: -67.35107011155344 Total: 33.0235; Orientation: 0.1500; Line Search: 25.3944
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 5f6b13f to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-67.35107011155344;dx=-9.885963892701756E-7
New Minimum: -67.35107011155344 > -75.0412937173311
END: th(1.3030021005312834E7)=-75.0412937173311; dx=-5.7492607979512223E-8 evalInputDelta=7.6902236057776605
Fitness changed from -67.35107011155344 to -75.0412937173311
Iteration 4 complete. Error: -75.0412937173311 Total: 20.2625; Orientation: 0.1484; Line Search: 12.7020
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 20fc23e7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.662e+04, gradient 7.682e-04, dot -0.692; [78348596-298e-4163-9aef-322c9874ba48 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -75.0412937173311, -67.35107011155344, -60.00329570711195, -49.384903693468345
LBFGS Accumulation History: 3 points
Removed measurement 20fc23e7 to history. Total: 3
th(0)=-75.0412937173311;dx=-5.899184258942312E-7
New Minimum: -75.0412937173311 > -79.41193806923765
WOLF (strong): th(2.8072329265690092E7)=-79.41193806923765; dx=9.124043802503299E-8 evalInputDelta=4.370644351906549
New Minimum: -79.41193806923765 > -79.97619611334423
END: th(1.4036164632845046E7)=-79.97619611334423; dx=-9.346038674064634E-8 evalInputDelta=4.934902396013129
Fitness changed from -75.0412937173311 to -79.97619611334423
Iteration 5 complete. Error: -79.97619611334423 Total: 30.7564; Orientation: 3.3650; Line Search: 19.7298
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 639a324d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.599e+04, gradient 7.399e-04, dot -0.681; [78348596-298e-4163-9aef-322c9874ba48 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -79.97619611334423, -67.35107011155344, -60.00329570711195, -49.384903693468345
LBFGS Accumulation History: 3 points
Removed measurement 639a324d to history. Total: 3
th(0)=-79.97619611334423;dx=-5.470368302683235E-7
New Minimum: -79.97619611334423 > -83.19073194814546
WOLF (strong): th(3.0240000000000007E7)=-83.19073194814546; dx=1.036151290579579E-7 evalInputDelta=3.214535834801225
New Minimum: -83.19073194814546 > -84.36575433880537
END: th(1.5120000000000004E7)=-84.36575433880537; dx=-3.581354685567798E-8 evalInputDelta=4.389558225461144
Fitness changed from -79.97619611334423 to -84.36575433880537
Iteration 6 complete. Error: -84.36575433880537 Total: 30.6813; Orientation: 2.9174; Line Search: 19.8540
Adding measurement 6b50cb3c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.087e+04, gradient 7.419e-04, dot -0.625; [78348596-298e-4163-9aef-322c9874ba48 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -84.36575433880537, -67.35107011155344, -60.00329570711195, -49.384903693468345
LBFGS Accumulation History: 3 points
Removed measurement 6b50cb3c to history. Total: 3
th(0)=-84.36575433880537;dx=-5.495962947568375E-7
New Minimum: -84.36575433880537 > -86.17163700814913
WOLF (strong): th(3.257505251328209E7)=-86.17163700814913; dx=1.5362737402004684E-7 evalInputDelta=1.8058826693437595
New Minimum: -86.17163700814913 > -88.53084479126129
WOLF (strong): th(1.6287526256641045E7)=-88.53084479126129; dx=2.6453081902402063E-8 evalInputDelta=4.165090452455914
END: th(5429175.418880348)=-87.06012456230083; dx=-3.4278037888628387E-7 evalInputDelta=2.6943702234954543
Fitness changed from -84.36575433880537 to -88.53084479126129
Iteration 7 complete. Error: -88.53084479126129 Total: 33.9938; Orientation: 2.5585; Line Search: 25.1657
Adding measurement 7dcf4926 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.085e+05, gradient 7.436e-04, dot -0.587; [78348596-298e-4163-9aef-322c9874ba48 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -88.53084479126129, -67.35107011155344, -60.00329570711195, -49.384903693468345
LBFGS Accumulation History: 3 points
Removed measurement 7dcf4926 to history. Total: 3
th(0)=-88.53084479126129;dx=-5.525835696603036E-7
New Minimum: -88.53084479126129 > -92.68406172512077
END: th(1.1696803860704206E7)=-92.68406172512077; dx=-8.457045453633956E-8 evalInputDelta=4.153216933859483
Fitness changed from -88.53084479126129 to -92.68406172512077
Iteration 8 complete. Error: -92.68406172512077 Total: 21.4014; Orientation: 2.5890; Line Search: 12.7090
Adding measurement 6ca83c6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.688e+04, gradient 5.640e-04, dot -0.714; [78348596-298e-4163-9aef-322c9874ba48 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -92.68406172512077, -67.35107011155344, -60.00329570711195, -49.384903693468345
LBFGS Accumulation History: 3 points
Removed measurement 6ca83c6 to history. Total: 3
th(0)=-92.68406172512077;dx=-3.175489349337581E-7
New Minimum: -92.68406172512077 > -96.68629255978033
END: th(2.5200000000000007E7)=-96.68629255978033; dx=-3.3280884474235187E-8 evalInputDelta=4.0022308346595565
Fitness changed from -92.68406172512077 to -96.68629255978033
Iteration 9 complete. Error: -96.68629255978033 Total: 21.5128; Orientation: 2.6579; Line Search: 12.7633
Adding measurement 6ea8a732 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.129e+05, gradient 7.305e-04, dot -0.518; [78348596-298e-4163-9aef-322c9874ba48 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -96.68629255978033, -67.35107011155344, -60.00329570711195, -49.384903693468345
LBFGS Accumulation History: 3 points
Removed measurement 6ea8a732 to history. Total: 3
th(0)=-96.68629255978033;dx=-5.335523016264056E-7
Armijo: th(5.429175418880349E7)=-91.04934575897029; dx=2.1942380690971755E-7 evalInputDelta=-5.636946800810037
New Minimum: -96.68629255978033 > -98.59712025324143
WOLF (strong): th(2.7145877094401743E7)=-98.59712025324143; dx=2.0930356205264691E-7 evalInputDelta=1.9108276934611013
New Minimum: -98.59712025324143 > -100.4206452740566
END: th(9048625.698133914)=-100.4206452740566; dx=-1.8766334032845039E-7 evalInputDelta=3.7343527142762696
Fitness changed from -96.68629255978033 to -100.4206452740566
Iteration 10 complete. Error: -100.4206452740566 Total: 33.3537; Orientation: 2.5956; Line Search: 24.6151
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -100.4206452740566 (> -Infinity) after 336.216s (< 5400.000s)
Returns
-100.4206452740566