BasicOptimizer.scala:89 executed in 299.36 seconds (6.648 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 588694622897700
Reset training subject: 588700477081300
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -37.26401159426685 > -37.31208905593057
th(0)=-37.31208905593057;dx=-1.95423023559834E-6
New Minimum: -37.31208905593057 > -37.31209108013372
END: th(2.154434690031884)=-37.31209108013372; dx=-1.7126407258437445E-6 evalInputDelta=2.0242031482098355E-6
Fitness changed from -37.26401159426685 to -37.31209108013372
Iteration 1 complete. Error: -37.31209108013372 Total: 30.1640; Orientation: 0.0933; Line Search: 12.2028
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 5e2356e8 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-37.31209108013372;dx=-1.722430997092819E-6
New Minimum: -37.31209108013372 > -37.31210120114781
WOLFE (weak): th(4.641588833612779)=-37.31210120114781; dx=-1.721260232482432E-6 evalInputDelta=1.012101409259003E-5
New Minimum: -37.31210120114781 > -37.31210929795711
WOLFE (weak): th(9.283177667225559)=-37.31210929795711; dx=-1.7200958232140387E-6 evalInputDelta=1.821782338851108E-5
New Minimum: -37.31210929795711 > -37.3121457335772
WOLFE (weak): th(27.849533001676676)=-37.3121457335772; dx=-1.719356120363736E-6 evalInputDelta=5.465344348465351E-5
New Minimum: -37.3121457335772 > -37.312303620853136
WOLFE (weak): th(111.3981320067067)=-37.312303620853136; dx=-1.718522213156932E-6 evalInputDelta=2.1254071941712027E-4
New Minimum: -37.312303620853136 > -37.3131436508956
WOLFE (weak): th(556.9906600335335)=-37.3131436508956; dx=-1.718470554337485E-6 evalInputDelta=0.0010525707618782576
New Minimum: -37.3131436508956 > -37.31839998780428
WOLFE (weak): th(3341.9439602012008)=-37.31839998780428; dx=-1.7181427862273709E-6 evalInputDelta=0.006308907670558028
New Minimum: -37.31839998780428 > -37.35621084063641
WOLFE (weak): th(23393.607721408407)=-37.35621084063641; dx=-1.715594533940703E-6 evalInputDelta=0.04411976050268862
New Minimum: -37.35621084063641 > -37.66313082774332
WOLFE (weak): th(187148.86177126726)=-37.66313082774332; dx=-1.6960531342958637E-6 evalInputDelta=0.35103974760960455
New Minimum: -37.66313082774332 > -40.28019065751298
END: th(1684339.7559414054)=-40.28019065751298; dx=-1.480584472407538E-6 evalInputDelta=2.968099577379263
Fitness changed from -37.31209108013372 to -40.28019065751298
Iteration 2 complete. Error: -40.28019065751298 Total: 70.2771; Orientation: 0.1423; Line Search: 62.8141
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 79e9b149 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-40.28019065751298;dx=-1.425518122213993E-6
New Minimum: -40.28019065751298 > -44.97133164254587
END: th(3628800.0000000005)=-44.97133164254587; dx=-9.690467340878065E-7 evalInputDelta=4.691140985032888
Fitness changed from -40.28019065751298 to -44.97133164254587
Iteration 3 complete. Error: -44.97133164254587 Total: 19.9062; Orientation: 0.1270; Line Search: 12.5658
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 1db5503b to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-44.97133164254587;dx=-9.316646299555721E-7
New Minimum: -44.97133164254587 > -51.072030656286564
END: th(7818012.603187701)=-51.072030656286564; dx=-5.203372395671313E-7 evalInputDelta=6.1006990137406945
Fitness changed from -44.97133164254587 to -51.072030656286564
Iteration 4 complete. Error: -51.072030656286564 Total: 19.4125; Orientation: 0.1300; Line Search: 12.1589
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 1203e755 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.402e+04, gradient 8.410e-04, dot -0.716; [0cb23f51-58a0-4d47-993b-1604a82866e1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -51.072030656286564, -44.97133164254587, -40.28019065751298, -37.31209108013372
LBFGS Accumulation History: 3 points
Removed measurement 1203e755 to history. Total: 3
th(0)=-51.072030656286564;dx=-7.069111952834812E-7
New Minimum: -51.072030656286564 > -57.656563800125895
END: th(1.6843397559414055E7)=-57.656563800125895; dx=-1.7193051464215901E-7 evalInputDelta=6.58453314383933
Fitness changed from -51.072030656286564 to -57.656563800125895
Iteration 5 complete. Error: -57.656563800125895 Total: 22.1377; Orientation: 2.6064; Line Search: 12.0832
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 71d9f19d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.543e+04, gradient 9.826e-04, dot -0.478; [0cb23f51-58a0-4d47-993b-1604a82866e1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -57.656563800125895, -44.97133164254587, -40.28019065751298, -37.31209108013372
LBFGS Accumulation History: 3 points
Removed measurement 71d9f19d to history. Total: 3
th(0)=-57.656563800125895;dx=-9.62999504384119E-7
Armijo: th(3.628800000000001E7)=-57.00907164362098; dx=1.8670712557566282E-7 evalInputDelta=-0.6474921565049172
New Minimum: -57.656563800125895 > -61.65253560345277
WOLF (strong): th(1.8144000000000004E7)=-61.65253560345277; dx=2.0130133064593207E-7 evalInputDelta=3.9959718033268743
New Minimum: -61.65253560345277 > -62.12596746461507
END: th(6048000.000000001)=-62.12596746461507; dx=-3.6552734949857035E-7 evalInputDelta=4.469403664489178
Fitness changed from -57.656563800125895 to -62.12596746461507
Iteration 6 complete. Error: -62.12596746461507 Total: 34.2803; Orientation: 2.5228; Line Search: 24.7443
Adding measurement 78b58cba to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.842e+04, gradient 7.088e-04, dot -0.678; [0cb23f51-58a0-4d47-993b-1604a82866e1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -62.12596746461507, -44.97133164254587, -40.28019065751298, -37.31209108013372
LBFGS Accumulation History: 3 points
Removed measurement 78b58cba to history. Total: 3
th(0)=-62.12596746461507;dx=-5.010992410461701E-7
New Minimum: -62.12596746461507 > -66.62060099235035
END: th(1.3030021005312836E7)=-66.62060099235035; dx=-1.9015700622992733E-7 evalInputDelta=4.4946335277352745
Fitness changed from -62.12596746461507 to -66.62060099235035
Iteration 7 complete. Error: -66.62060099235035 Total: 21.3736; Orientation: 2.4502; Line Search: 12.5090
Adding measurement 22ec3605 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.852e+04, gradient 7.973e-04, dot -0.574; [0cb23f51-58a0-4d47-993b-1604a82866e1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -66.62060099235035, -44.97133164254587, -40.28019065751298, -37.31209108013372
LBFGS Accumulation History: 3 points
Removed measurement 22ec3605 to history. Total: 3
th(0)=-66.62060099235035;dx=-6.345518395076849E-7
New Minimum: -66.62060099235035 > -68.62680627631859
WOLF (strong): th(2.8072329265690096E7)=-68.62680627631859; dx=1.0139300780818501E-7 evalInputDelta=2.0062052839682423
New Minimum: -68.62680627631859 > -70.0764403148249
WOLF (strong): th(1.4036164632845048E7)=-70.0764403148249; dx=1.9164179615412492E-8 evalInputDelta=3.455839322474546
END: th(4678721.544281683)=-69.08300225128629; dx=-3.273780226494366E-7 evalInputDelta=2.462401258935941
Fitness changed from -66.62060099235035 to -70.0764403148249
Iteration 8 complete. Error: -70.0764403148249 Total: 33.5952; Orientation: 2.4352; Line Search: 24.9427
Adding measurement 4c36a67a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.053e+04, gradient 9.020e-04, dot -0.462; [0cb23f51-58a0-4d47-993b-1604a82866e1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -70.0764403148249, -44.97133164254587, -40.28019065751298, -37.31209108013372
LBFGS Accumulation History: 3 points
Removed measurement 4c36a67a to history. Total: 3
th(0)=-70.0764403148249;dx=-8.133043622297757E-7
New Minimum: -70.0764403148249 > -74.74120674467312
END: th(1.0080000000000004E7)=-74.74120674467312; dx=-1.6767669047505372E-8 evalInputDelta=4.664766429848228
Fitness changed from -70.0764403148249 to -74.74120674467312
Iteration 9 complete. Error: -74.74120674467312 Total: 20.7206; Orientation: 2.4767; Line Search: 12.2569
Adding measurement 2923b833 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.559e+04, gradient 7.214e-04, dot -0.578; [0cb23f51-58a0-4d47-993b-1604a82866e1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -74.74120674467312, -44.97133164254587, -40.28019065751298, -37.31209108013372
LBFGS Accumulation History: 3 points
Removed measurement 2923b833 to history. Total: 3
th(0)=-74.74120674467312;dx=-5.19600876750891E-7
New Minimum: -74.74120674467312 > -77.33735048204888
WOLF (strong): th(2.1716701675521396E7)=-77.33735048204888; dx=1.0204994974270202E-7 evalInputDelta=2.5961437373757548
New Minimum: -77.33735048204888 > -77.96220240125888
END: th(1.0858350837760698E7)=-77.96220240125888; dx=-6.789664490113061E-8 evalInputDelta=3.2209956565857567
Fitness changed from -74.74120674467312 to -77.96220240125888
Iteration 10 complete. Error: -77.96220240125888 Total: 26.2559; Orientation: 2.4073; Line Search: 17.9525
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -77.96220240125888 (> -Infinity) after 299.359s (< 5400.000s)
Returns
-77.96220240125888