BasicOptimizer.scala:89 executed in 134.87 seconds (1.679 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 73127158071500
Reset training subject: 73129821511000
Adding measurement 2187c66f to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-18.557078673832393;dx=-7.662170599788089E-7
New Minimum: -18.557078673832393 > -18.557079369354515
END: th(2.154434690031884)=-18.557079369354515; dx=-6.562721038314618E-7 evalInputDelta=6.955221216742302E-7
Fitness changed from -18.557078673832393 to -18.557079369354515
Iteration 1 complete. Error: -18.557079369354515 Total: 13.0153; Orientation: 0.0604; Line Search: 5.2826
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 354e555e to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-18.557079369354515;dx=-6.5446391197789E-7
New Minimum: -18.557079369354515 > -18.55708354248671
WOLFE (weak): th(4.641588833612779)=-18.55708354248671; dx=-6.543074337520035E-7 evalInputDelta=4.173132193585616E-6
New Minimum: -18.55708354248671 > -18.557087020096155
WOLFE (weak): th(9.283177667225559)=-18.557087020096155; dx=-6.538635178107143E-7 evalInputDelta=7.650741640219394E-6
New Minimum: -18.557087020096155 > -18.557100930527415
WOLFE (weak): th(27.849533001676676)=-18.557100930527415; dx=-6.532885102244331E-7 evalInputDelta=2.156117290041948E-5
New Minimum: -18.557100930527415 > -18.557162136301084
WOLFE (weak): th(111.3981320067067)=-18.557162136301084; dx=-6.534993220567155E-7 evalInputDelta=8.27669465692793E-5
New Minimum: -18.557162136301084 > -18.557492504889314
WOLFE (weak): th(556.9906600335335)=-18.557492504889314; dx=-6.533193323877742E-7 evalInputDelta=4.1313553479938037E-4
New Minimum: -18.557492504889314 > -18.559547613176175
WOLFE (weak): th(3341.9439602012008)=-18.559547613176175; dx=-6.528652044020367E-7 evalInputDelta=0.00246824382166011
New Minimum: -18.559547613176175 > -18.57431405473158
WOLFE (weak): th(23393.607721408407)=-18.57431405473158; dx=-6.49988259282153E-7 evalInputDelta=0.017234685377065517
New Minimum: -18.57431405473158 > -18.691845420069225
WOLFE (weak): th(187148.86177126726)=-18.691845420069225; dx=-6.168838257089323E-7 evalInputDelta=0.1347660507147097
New Minimum: -18.691845420069225 > -19.395492248792813
END: th(1684339.7559414054)=-19.395492248792813; dx=-2.639089357717339E-7 evalInputDelta=0.8384128794382981
Fitness changed from -18.557079369354515 to -19.395492248792813
Iteration 2 complete. Error: -19.395492248792813 Total: 29.4096; Orientation: 0.0584; Line Search: 26.2188
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 85290fe to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-19.395492248792813;dx=-6.345248284363546E-7
New Minimum: -19.395492248792813 > -20.284584456926066
END: th(3628800.0000000005)=-20.284584456926066; dx=-1.013663863726492E-7 evalInputDelta=0.8890922081332526
Fitness changed from -19.395492248792813 to -20.284584456926066
Iteration 3 complete. Error: -20.284584456926066 Total: 8.5156; Orientation: 0.0592; Line Search: 5.2813
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 1b32a42b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.772e+03, gradient 7.893e-04, dot -0.522; [653145d2-e7bc-44e2-b246-666f22fe7c08 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -20.284584456926066, -19.395492248792813, -18.557079369354515, -18.557078673832393
LBFGS Accumulation History: 3 points
Removed measurement 1b32a42b to history. Total: 3
th(0)=-20.284584456926066;dx=-6.227665159596337E-7
New Minimum: -20.284584456926066 > -21.509540955923704
END: th(7818012.603187701)=-21.509540955923704; dx=-2.2246224308861093E-8 evalInputDelta=1.2249564989976385
Fitness changed from -20.284584456926066 to -21.509540955923704
Iteration 4 complete. Error: -21.509540955923704 Total: 9.3282; Orientation: 1.0344; Line Search: 5.2960
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 71f1b1fe to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.687e+03, gradient 7.236e-04, dot -0.227; [653145d2-e7bc-44e2-b246-666f22fe7c08 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -21.509540955923704, -19.395492248792813, -18.557079369354515, -18.557078673832393
LBFGS Accumulation History: 3 points
Removed measurement 71f1b1fe to history. Total: 3
th(0)=-21.509540955923704;dx=-5.230529396919334E-7
New Minimum: -21.509540955923704 > -23.5198738594954
WOLF (strong): th(1.6843397559414055E7)=-23.5198738594954; dx=6.76167321724743E-9 evalInputDelta=2.0103329035716975
New Minimum: -23.5198738594954 > -23.650256580231005
END: th(8421698.779707028)=-23.650256580231005; dx=-1.0414998529488084E-8 evalInputDelta=2.140715624307301
Fitness changed from -21.509540955923704 to -23.650256580231005
Iteration 5 complete. Error: -23.650256580231005 Total: 12.0957; Orientation: 1.0257; Line Search: 7.9215
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 7b2de6e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.134e+03, gradient 7.261e-04, dot -0.304; [653145d2-e7bc-44e2-b246-666f22fe7c08 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -23.650256580231005, -19.395492248792813, -18.557079369354515, -18.557078673832393
LBFGS Accumulation History: 3 points
Removed measurement 7b2de6e to history. Total: 3
th(0)=-23.650256580231005;dx=-5.271642166801593E-7
New Minimum: -23.650256580231005 > -24.502715008632165
WOLF (strong): th(1.8144000000000004E7)=-24.502715008632165; dx=3.164945339646E-8 evalInputDelta=0.8524584284011603
New Minimum: -24.502715008632165 > -25.175660323402603
WOLF (strong): th(9072000.000000002)=-25.175660323402603; dx=8.35053623797652E-8 evalInputDelta=1.5254037431715979
END: th(3024000.0000000005)=-25.101083214682813; dx=-2.592069697354411E-7 evalInputDelta=1.4508266344518077
Fitness changed from -23.650256580231005 to -25.175660323402603
Iteration 6 complete. Error: -25.175660323402603 Total: 14.3720; Orientation: 0.9705; Line Search: 10.3502
Adding measurement 46dc86c4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.168e+03, gradient 7.116e-04, dot -0.289; [653145d2-e7bc-44e2-b246-666f22fe7c08 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -25.175660323402603, -19.395492248792813, -18.557079369354515, -18.557078673832393
LBFGS Accumulation History: 3 points
Removed measurement 46dc86c4 to history. Total: 3
th(0)=-25.175660323402603;dx=-5.062908476946249E-7
New Minimum: -25.175660323402603 > -27.076058564174573
WOLF (strong): th(6515010.502656418)=-27.076058564174573; dx=3.2214561758434535E-8 evalInputDelta=1.9003982407719704
END: th(3257505.251328209)=-26.721964830413057; dx=-2.6678281507578685E-7 evalInputDelta=1.5463045070104542
Fitness changed from -25.175660323402603 to -27.076058564174573
Iteration 7 complete. Error: -27.076058564174573 Total: 11.4675; Orientation: 0.9561; Line Search: 7.7640
Adding measurement 35d327ef to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.498e+04, gradient 6.748e-04, dot -0.369; [653145d2-e7bc-44e2-b246-666f22fe7c08 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -27.076058564174573, -19.395492248792813, -18.557079369354515, -18.557078673832393
LBFGS Accumulation History: 3 points
Removed measurement 35d327ef to history. Total: 3
th(0)=-27.076058564174573;dx=-4.551853176501585E-7
New Minimum: -27.076058564174573 > -27.828623656917898
WOLF (strong): th(7018082.316422524)=-27.828623656917898; dx=1.291434672816858E-7 evalInputDelta=0.7525650927433247
New Minimum: -27.828623656917898 > -28.182569614441533
END: th(3509041.158211262)=-28.182569614441533; dx=-3.61238971095921E-8 evalInputDelta=1.1065110502669597
Fitness changed from -27.076058564174573 to -28.182569614441533
Iteration 8 complete. Error: -28.182569614441533 Total: 11.2653; Orientation: 0.9449; Line Search: 7.7839
Adding measurement 769aeb7d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.065e+04, gradient 5.916e-04, dot -0.471; [653145d2-e7bc-44e2-b246-666f22fe7c08 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -28.182569614441533, -19.395492248792813, -18.557079369354515, -18.557078673832393
LBFGS Accumulation History: 3 points
Removed measurement 769aeb7d to history. Total: 3
th(0)=-28.182569614441533;dx=-3.4975973825252385E-7
New Minimum: -28.182569614441533 > -28.408135547022834
WOLF (strong): th(7560000.000000003)=-28.408135547022834; dx=4.530255535164211E-8 evalInputDelta=0.22556593258130064
New Minimum: -28.408135547022834 > -28.66665989838176
WOLF (strong): th(3780000.0000000014)=-28.66665989838176; dx=4.720136422585773E-8 evalInputDelta=0.48409028394022613
END: th(1260000.0000000005)=-28.58020423207816; dx=-1.869152228310476E-7 evalInputDelta=0.39763461763662633
Fitness changed from -28.182569614441533 to -28.66665989838176
Iteration 9 complete. Error: -28.66665989838176 Total: 13.8178; Orientation: 1.0133; Line Search: 10.3180
Adding measurement 78fc3b39 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.081e+04, gradient 6.406e-04, dot -0.417; [653145d2-e7bc-44e2-b246-666f22fe7c08 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -28.66665989838176, -19.395492248792813, -18.557079369354515, -18.557078673832393
LBFGS Accumulation History: 3 points
Removed measurement 78fc3b39 to history. Total: 3
th(0)=-28.66665989838176;dx=-4.099962238901026E-7
New Minimum: -28.66665989838176 > -29.325043936988028
WOLF (strong): th(2714587.7094401745)=-29.325043936988028; dx=4.533093262570084E-8 evalInputDelta=0.6583840386062683
END: th(1357293.8547200873)=-29.193950166167607; dx=-2.425502996702321E-7 evalInputDelta=0.5272902677858475
Fitness changed from -28.66665989838176 to -29.325043936988028
Iteration 10 complete. Error: -29.325043936988028 Total: 11.1232; Orientation: 0.9487; Line Search: 7.7804
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -29.325043936988028 (> -Infinity) after 134.865s (< 5400.000s)
Returns
-29.325043936988028