BasicOptimizer.scala:89 executed in 206.54 seconds (2.519 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 19558155450254
Reset training subject: 19562880920300
Adding measurement 23f5262e to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.1336328685283661;dx=-6.270798412772556E-7
New Minimum: -0.1336328685283661 > -0.13363337516784668
WOLFE (weak): th(2.154434690031884)=-0.13363337516784668; dx=-5.867143591150116E-7 evalInputDelta=5.066394805908203E-7
New Minimum: -0.13363337516784668 > -0.1336340308189392
WOLFE (weak): th(4.308869380063768)=-0.1336340308189392; dx=-5.866568010817954E-7 evalInputDelta=1.1622905731201172E-6
New Minimum: -0.1336340308189392 > -0.13363665342330933
WOLFE (weak): th(12.926608140191302)=-0.13363665342330933; dx=-5.866564308240364E-7 evalInputDelta=3.7848949432373047E-6
New Minimum: -0.13363665342330933 > -0.1336480677127838
WOLFE (weak): th(51.70643256076521)=-0.1336480677127838; dx=-5.866442009792335E-7 evalInputDelta=1.519918441772461E-5
New Minimum: -0.1336480677127838 > -0.13370871543884277
WOLFE (weak): th(258.53216280382605)=-0.13370871543884277; dx=-5.866266641975676E-7 evalInputDelta=7.584691047668457E-5
New Minimum: -0.13370871543884277 > -0.13408780097961426
WOLFE (weak): th(1551.1929768229563)=-0.13408780097961426; dx=-5.864160008921476E-7 evalInputDelta=4.5493245124816895E-4
New Minimum: -0.13408780097961426 > -0.1368137001991272
WOLFE (weak): th(10858.350837760694)=-0.1368137001991272; dx=-5.851645538023854E-7 evalInputDelta=0.0031808316707611084
New Minimum: -0.1368137001991272 > -0.15885737538337708
WOLFE (weak): th(86866.80670208555)=-0.15885737538337708; dx=-5.744707611293577E-7 evalInputDelta=0.025224506855010986
New Minimum: -0.15885737538337708 > -0.3387911021709442
END: th(781801.26031877)=-0.3387911021709442; dx=-4.5926211300709505E-7 evalInputDelta=0.20515823364257812
Fitness changed from -0.1336328685283661 to -0.3387911021709442
Iteration 1 complete. Error: -0.3387911021709442 Total: 56.3287; Orientation: 0.0478; Line Search: 42.8900
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 76c477bd to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-0.3387911021709442;dx=-4.1000140662972337E-7
New Minimum: -0.3387911021709442 > -0.6243501901626587
END: th(1684339.7559414052)=-0.6243501901626587; dx=-2.7141667853720036E-7 evalInputDelta=0.2855590879917145
Fitness changed from -0.3387911021709442 to -0.6243501901626587
Iteration 2 complete. Error: -0.6243501901626587 Total: 12.6556; Orientation: 0.0450; Line Search: 8.6040
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 650fa8b4 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-0.6243501901626587;dx=-2.3670781173334117E-7
New Minimum: -0.6243501901626587 > -0.9542914628982544
END: th(3628800.0)=-0.9542914628982544; dx=-1.3238081699660644E-7 evalInputDelta=0.3299412727355957
Fitness changed from -0.6243501901626587 to -0.9542914628982544
Iteration 3 complete. Error: -0.9542914628982544 Total: 13.4550; Orientation: 0.0556; Line Search: 8.8979
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 3cf3ff97 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.211e+03, gradient 3.545e-04, dot -0.845; [96d0ed01-02c2-4eaa-afac-9c915d81dbc8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.9542914628982544, -0.6243501901626587, -0.3387911021709442, -0.1336328685283661
LBFGS Accumulation History: 3 points
Removed measurement 3cf3ff97 to history. Total: 3
th(0)=-0.9542914628982544;dx=-1.2564421426446162E-7
New Minimum: -0.9542914628982544 > -1.2825119495391846
END: th(7818012.6031877)=-1.2825119495391846; dx=-5.120192138201814E-8 evalInputDelta=0.3282204866409302
Fitness changed from -0.9542914628982544 to -1.2825119495391846
Iteration 4 complete. Error: -1.2825119495391846 Total: 14.5822; Orientation: 1.0572; Line Search: 9.0229
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 4642dbb1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.609e+03, gradient 2.775e-04, dot -0.649; [96d0ed01-02c2-4eaa-afac-9c915d81dbc8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.2825119495391846, -0.6243501901626587, -0.3387911021709442, -0.1336328685283661
LBFGS Accumulation History: 3 points
Removed measurement 4642dbb1 to history. Total: 3
th(0)=-1.2825119495391846;dx=-7.699786364721462E-8
New Minimum: -1.2825119495391846 > -1.5216721296310425
END: th(1.684339755941405E7)=-1.5216721296310425; dx=-4.056279050971824E-9 evalInputDelta=0.2391601800918579
Fitness changed from -1.2825119495391846 to -1.5216721296310425
Iteration 5 complete. Error: -1.5216721296310425 Total: 14.3988; Orientation: 1.0032; Line Search: 8.7859
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 2e38f643 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.743e+03, gradient 2.894e-04, dot -0.390; [96d0ed01-02c2-4eaa-afac-9c915d81dbc8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.5216721296310425, -0.6243501901626587, -0.3387911021709442, -0.1336328685283661
LBFGS Accumulation History: 3 points
Removed measurement 2e38f643 to history. Total: 3
th(0)=-1.5216721296310425;dx=-8.37375354875469E-8
Armijo: th(3.6288E7)=-1.3301664292812347; dx=3.760147652109235E-8 evalInputDelta=-0.19150570034980774
New Minimum: -1.5216721296310425 > -1.6225803047418594
WOLF (strong): th(1.8144E7)=-1.6225803047418594; dx=2.4285709704796594E-8 evalInputDelta=0.10090817511081696
New Minimum: -1.6225803047418594 > -1.6686513423919678
END: th(6048000.0)=-1.6686513423919678; dx=-2.0192364495543646E-8 evalInputDelta=0.1469792127609253
Fitness changed from -1.5216721296310425 to -1.6686513423919678
Iteration 6 complete. Error: -1.6686513423919678 Total: 22.3475; Orientation: 0.9966; Line Search: 17.3636
Adding measurement 31d6785f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.385e+03, gradient 1.958e-04, dot -0.487; [96d0ed01-02c2-4eaa-afac-9c915d81dbc8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.6686513423919678, -0.6243501901626587, -0.3387911021709442, -0.1336328685283661
LBFGS Accumulation History: 3 points
Removed measurement 31d6785f to history. Total: 3
th(0)=-1.6686513423919678;dx=-3.834516375316237E-8
New Minimum: -1.6686513423919678 > -1.78251214325428
END: th(1.3030021005312834E7)=-1.78251214325428; dx=-1.5582692378977295E-9 evalInputDelta=0.11386080086231232
Fitness changed from -1.6686513423919678 to -1.78251214325428
Iteration 7 complete. Error: -1.78251214325428 Total: 13.2995; Orientation: 1.0414; Line Search: 8.0741
Adding measurement 7f05b4bd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.590e+03, gradient 2.319e-04, dot -0.388; [96d0ed01-02c2-4eaa-afac-9c915d81dbc8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.78251214325428, -0.6243501901626587, -0.3387911021709442, -0.1336328685283661
LBFGS Accumulation History: 3 points
Removed measurement 7f05b4bd to history. Total: 3
th(0)=-1.78251214325428;dx=-5.378728364123745E-8
Armijo: th(2.8072329265690092E7)=-1.70095694065094; dx=2.222235638996389E-8 evalInputDelta=-0.08155520260334015
New Minimum: -1.78251214325428 > -1.8313555270433426
WOLF (strong): th(1.4036164632845046E7)=-1.8313555270433426; dx=1.3774645630890119E-8 evalInputDelta=0.0488433837890625
New Minimum: -1.8313555270433426 > -1.8537656515836716
END: th(4678721.544281682)=-1.8537656515836716; dx=-1.1060533182931501E-8 evalInputDelta=0.07125350832939148
Fitness changed from -1.78251214325428 to -1.8537656515836716
Iteration 8 complete. Error: -1.8537656515836716 Total: 23.2379; Orientation: 1.0301; Line Search: 18.1269
Adding measurement 7c780a15 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.905e+03, gradient 1.535e-04, dot -0.534; [96d0ed01-02c2-4eaa-afac-9c915d81dbc8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.8537656515836716, -0.6243501901626587, -0.3387911021709442, -0.1336328685283661
LBFGS Accumulation History: 3 points
Removed measurement 7c780a15 to history. Total: 3
th(0)=-1.8537656515836716;dx=-2.356900997865725E-8
New Minimum: -1.8537656515836716 > -1.9164111614227295
END: th(1.0080000000000002E7)=-1.9164111614227295; dx=-3.897237960549962E-9 evalInputDelta=0.06264550983905792
Fitness changed from -1.8537656515836716 to -1.9164111614227295
Iteration 9 complete. Error: -1.9164111614227295 Total: 13.4461; Orientation: 0.9998; Line Search: 8.5057
Adding measurement c1837ff to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.987e+03, gradient 1.830e-04, dot -0.427; [96d0ed01-02c2-4eaa-afac-9c915d81dbc8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.9164111614227295, -0.6243501901626587, -0.3387911021709442, -0.1336328685283661
LBFGS Accumulation History: 3 points
Removed measurement c1837ff to history. Total: 3
th(0)=-1.9164111614227295;dx=-3.349699761675387E-8
Armijo: th(2.1716701675521392E7)=-1.9003286510705948; dx=1.1382266858229397E-8 evalInputDelta=-0.016082510352134705
New Minimum: -1.9164111614227295 > -1.9510018974542618
WOLF (strong): th(1.0858350837760696E7)=-1.9510018974542618; dx=6.539295442741567E-9 evalInputDelta=0.03459073603153229
New Minimum: -1.9510018974542618 > -1.9544700533151627
END: th(3619450.2792535652)=-1.9544700533151627; dx=-9.878125384529704E-9 evalInputDelta=0.038058891892433167
Fitness changed from -1.9164111614227295 to -1.9544700533151627
Iteration 10 complete. Error: -1.9544700533151627 Total: 22.7215; Orientation: 1.1365; Line Search: 16.8294
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -1.9544700533151627 (> -Infinity) after 206.544s (< 5400.000s)
Returns
-1.9544700533151627