BasicOptimizer.scala:89 executed in 44.53 seconds (0.766 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 215333287902800
Reset training subject: 215333893106800
Adding measurement 15ee97a5 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-2.0846149085976284;dx=-1.4046370143013527E-5
New Minimum: -2.0846149085976284 > -2.08463728639889
WOLFE (weak): th(2.154434690031884)=-2.08463728639889; dx=-1.4050152446042912E-5 evalInputDelta=2.2377801261352914E-5
New Minimum: -2.08463728639889 > -2.084660120545551
WOLFE (weak): th(4.308869380063768)=-2.084660120545551; dx=-1.4022594634301465E-5 evalInputDelta=4.521194792239669E-5
New Minimum: -2.084660120545551 > -2.0847503119881847
WOLFE (weak): th(12.926608140191302)=-2.0847503119881847; dx=-1.4012564822455131E-5 evalInputDelta=1.3540339055628792E-4
New Minimum: -2.0847503119881847 > -2.0851571330235004
WOLFE (weak): th(51.70643256076521)=-2.0851571330235004; dx=-1.4022974933846625E-5 evalInputDelta=5.422244258719289E-4
New Minimum: -2.0851571330235004 > -2.087326032181084
WOLFE (weak): th(258.53216280382605)=-2.087326032181084; dx=-1.4082069907381082E-5 evalInputDelta=0.002711123583455599
New Minimum: -2.087326032181084 > -2.100812459451185
WOLFE (weak): th(1551.1929768229563)=-2.100812459451185; dx=-1.3904917499417113E-5 evalInputDelta=0.016197550853556564
New Minimum: -2.100812459451185 > -2.19426053659149
WOLFE (weak): th(10858.350837760694)=-2.19426053659149; dx=-1.298815816553734E-5 evalInputDelta=0.10964562799386135
New Minimum: -2.19426053659149 > -2.787960748187886
END: th(86866.80670208555)=-2.787960748187886; dx=-8.376660978392719E-6 evalInputDelta=0.7033458395902574
Fitness changed from -2.0846149085976284 to -2.787960748187886
Iteration 1 complete. Error: -2.787960748187886 Total: 7.3788; Orientation: 0.0163; Line Search: 5.4232
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 7c915900 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-2.787960748187886;dx=-1.4582803508685252E-5
New Minimum: -2.787960748187886 > -3.816194199314987
END: th(187148.86177126726)=-3.816194199314987; dx=-4.581007070608874E-6 evalInputDelta=1.0282334511271012
Fitness changed from -2.787960748187886 to -3.816194199314987
Iteration 2 complete. Error: -3.816194199314987 Total: 1.8045; Orientation: 0.0160; Line Search: 1.2020
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 4aa7bc9a to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-3.816194199314987;dx=-1.4226215519690623E-5
Armijo: th(403200.00000000006)=-3.1087377259695606; dx=-1.2825023839111591E-6 evalInputDelta=-0.7074564733454265
New Minimum: -3.816194199314987 > -4.204839916781854
END: th(201600.00000000003)=-4.204839916781854; dx=-3.899297290167968E-6 evalInputDelta=0.3886457174668667
Fitness changed from -3.816194199314987 to -4.204839916781854
Iteration 3 complete. Error: -4.204839916781854 Total: 2.4423; Orientation: 0.0166; Line Search: 1.8461
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 11351a0d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.155e+03, gradient 3.909e-03, dot -0.294; [b59dae98-ba75-4ec7-a7af-0fa8823b7e50 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.204839916781854, -3.816194199314987, -2.787960748187886, -2.0846149085976284
LBFGS Accumulation History: 3 points
Removed measurement 11351a0d to history. Total: 3
th(0)=-4.204839916781854;dx=-1.5232693654605927E-5
Armijo: th(434334.03351042786)=-2.8048381539536127; dx=-1.134475692605992E-6 evalInputDelta=-1.400001762828241
New Minimum: -4.204839916781854 > -4.397724722413821
END: th(217167.01675521393)=-4.397724722413821; dx=-4.292172294757429E-6 evalInputDelta=0.19288480563196764
Fitness changed from -4.204839916781854 to -4.397724722413821
Iteration 4 complete. Error: -4.397724722413821 Total: 2.9132; Orientation: 0.2963; Line Search: 2.0386
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 201ce325 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.274e+03, gradient 3.987e-03, dot -0.284; [b59dae98-ba75-4ec7-a7af-0fa8823b7e50 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.397724722413821, -3.816194199314987, -2.787960748187886, -2.0846149085976284
LBFGS Accumulation History: 3 points
Removed measurement 201ce325 to history. Total: 3
th(0)=-4.397724722413821;dx=-1.5765640366186454E-5
Armijo: th(467872.15442816826)=-2.3332899779374774; dx=-1.1147923226923762E-6 evalInputDelta=-2.064434744476344
Armijo: th(233936.07721408413)=-4.322727240708811; dx=-4.287499225456718E-6 evalInputDelta=-0.07499748170501075
New Minimum: -4.397724722413821 > -4.9200648699571605
END: th(77978.69240469471)=-4.9200648699571605; dx=-1.1092014723994528E-5 evalInputDelta=0.5223401475433391
Fitness changed from -4.397724722413821 to -4.9200648699571605
Iteration 5 complete. Error: -4.9200648699571605 Total: 3.3308; Orientation: 0.2952; Line Search: 2.4441
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 3996a52f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.187e+03, gradient 3.569e-03, dot -0.321; [b59dae98-ba75-4ec7-a7af-0fa8823b7e50 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.9200648699571605, -3.816194199314987, -2.787960748187886, -2.0846149085976284
LBFGS Accumulation History: 3 points
Removed measurement 3996a52f to history. Total: 3
th(0)=-4.9200648699571605;dx=-1.2684860684569145E-5
New Minimum: -4.9200648699571605 > -5.2137506895037875
END: th(168000.00000000006)=-5.2137506895037875; dx=-7.121889198266044E-6 evalInputDelta=0.293685819546627
Fitness changed from -4.9200648699571605 to -5.2137506895037875
Iteration 6 complete. Error: -5.2137506895037875 Total: 2.0669; Orientation: 0.2989; Line Search: 1.1833
Adding measurement 438c2075 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.199e+03, gradient 3.705e-03, dot -0.300; [b59dae98-ba75-4ec7-a7af-0fa8823b7e50 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.2137506895037875, -3.816194199314987, -2.787960748187886, -2.0846149085976284
LBFGS Accumulation History: 3 points
Removed measurement 438c2075 to history. Total: 3
th(0)=-5.2137506895037875;dx=-1.370343867571004E-5
Armijo: th(361945.0279253566)=-4.194048450223996; dx=-3.1406145695224045E-6 evalInputDelta=-1.0197022392797912
New Minimum: -5.2137506895037875 > -5.450798537678759
END: th(180972.5139626783)=-5.450798537678759; dx=-6.721553805813956E-6 evalInputDelta=0.2370478481749716
Fitness changed from -5.2137506895037875 to -5.450798537678759
Iteration 7 complete. Error: -5.450798537678759 Total: 2.6990; Orientation: 0.3003; Line Search: 1.8604
Adding measurement d442465 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.727e+03, gradient 3.665e-03, dot -0.280; [b59dae98-ba75-4ec7-a7af-0fa8823b7e50 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.450798537678759, -3.816194199314987, -2.787960748187886, -2.0846149085976284
LBFGS Accumulation History: 3 points
Removed measurement d442465 to history. Total: 3
th(0)=-5.450798537678759;dx=-1.3369446761005584E-5
Armijo: th(389893.4620234736)=-4.055005307545371; dx=-2.6814159768878402E-6 evalInputDelta=-1.3957932301333882
New Minimum: -5.450798537678759 > -5.567474721321847
END: th(194946.7310117368)=-5.567474721321847; dx=-6.379507067724558E-6 evalInputDelta=0.1166761836430883
Fitness changed from -5.450798537678759 to -5.567474721321847
Iteration 8 complete. Error: -5.567474721321847 Total: 2.8529; Orientation: 0.2943; Line Search: 2.0063
Adding measurement 7e27d5a8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.413e+03, gradient 3.839e-03, dot -0.257; [b59dae98-ba75-4ec7-a7af-0fa8823b7e50 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.567474721321847, -3.816194199314987, -2.787960748187886, -2.0846149085976284
LBFGS Accumulation History: 3 points
Removed measurement 7e27d5a8 to history. Total: 3
th(0)=-5.567474721321847;dx=-1.4685172284441016E-5
Armijo: th(420000.0000000002)=-3.765424225689582; dx=-2.1450045920974274E-6 evalInputDelta=-1.8020504956322654
New Minimum: -5.567474721321847 > -5.651665624852129
END: th(210000.0000000001)=-5.651665624852129; dx=-5.930470845366708E-6 evalInputDelta=0.08419090353028125
Fitness changed from -5.567474721321847 to -5.651665624852129
Iteration 9 complete. Error: -5.651665624852129 Total: 2.7438; Orientation: 0.2819; Line Search: 1.8792
Adding measurement 4e685719 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.819e+03, gradient 3.941e-03, dot -0.237; [b59dae98-ba75-4ec7-a7af-0fa8823b7e50 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.651665624852129, -3.816194199314987, -2.787960748187886, -2.0846149085976284
LBFGS Accumulation History: 3 points
Removed measurement 4e685719 to history. Total: 3
th(0)=-5.651665624852129;dx=-1.530425813301321E-5
Armijo: th(452431.2849066958)=-3.396866151845602; dx=-1.8889461932718715E-6 evalInputDelta=-2.2547994730065266
New Minimum: -5.651665624852129 > -5.688396111593729
END: th(226215.6424533479)=-5.688396111593729; dx=-5.55148658842383E-6 evalInputDelta=0.03673048674160029
Fitness changed from -5.651665624852129 to -5.688396111593729
Iteration 10 complete. Error: -5.688396111593729 Total: 2.6922; Orientation: 0.2923; Line Search: 1.8444
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 253b9b89 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.520e+03, gradient 3.938e-03, dot -0.228; [b59dae98-ba75-4ec7-a7af-0fa8823b7e50 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.688396111593729, -3.816194199314987, -2.787960748187886, -2.0846149085976284
LBFGS Accumulation History: 3 points
Removed measurement 253b9b89 to history. Total: 3
th(0)=-5.688396111593729;dx=-1.5504140221654372E-5
Armijo: th(487366.82752934203)=-3.1564649032632346; dx=-1.3817464406777544E-6 evalInputDelta=-2.5319312083304943
New Minimum: -5.688396111593729 > -5.704043510580984
END: th(243683.41376467102)=-5.704043510580984; dx=-5.5675820425124115E-6 evalInputDelta=0.0156473989872552
Fitness changed from -5.688396111593729 to -5.704043510580984
Iteration 11 complete. Error: -5.704043510580984 Total: 2.6756; Orientation: 0.2963; Line Search: 1.8042
Adding measurement 1db87998 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.603e+03, gradient 3.949e-03, dot -0.216; [b59dae98-ba75-4ec7-a7af-0fa8823b7e50 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.704043510580984, -3.816194199314987, -2.787960748187886, -2.0846149085976284
LBFGS Accumulation History: 3 points
Removed measurement 1db87998 to history. Total: 3
th(0)=-5.704043510580984;dx=-1.549946374452156E-5
Armijo: th(525000.0000000002)=-2.8884592642976603; dx=-1.6039274009603684E-6 evalInputDelta=-2.815584246283324
Armijo: th(262500.0000000001)=-5.580670817107437; dx=-5.446978550163084E-6 evalInputDelta=-0.12337269347354685
New Minimum: -5.704043510580984 > -6.220431627125471
END: th(87500.00000000004)=-6.220431627125471; dx=-1.1651663810662204E-5 evalInputDelta=0.516388116544487
Fitness changed from -5.704043510580984 to -6.220431627125471
Iteration 12 complete. Error: -6.220431627125471 Total: 3.5058; Orientation: 0.2964; Line Search: 2.6598
Adding measurement 72cee5f7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.150e+04, gradient 3.585e-03, dot -0.241; [b59dae98-ba75-4ec7-a7af-0fa8823b7e50 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.220431627125471, -3.816194199314987, -2.787960748187886, -2.0846149085976284
LBFGS Accumulation History: 3 points
Removed measurement 72cee5f7 to history. Total: 3
th(0)=-6.220431627125471;dx=-1.2756171766247321E-5
New Minimum: -6.220431627125471 > -6.475286341598592
END: th(188513.03537778993)=-6.475286341598592; dx=-7.761689369322869E-6 evalInputDelta=0.2548547144731206
Fitness changed from -6.220431627125471 to -6.475286341598592
Iteration 13 complete. Error: -6.475286341598592 Total: 2.0749; Orientation: 0.2882; Line Search: 1.2152
Adding measurement 3d3f783 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.457e+04, gradient 3.526e-03, dot -0.245; [b59dae98-ba75-4ec7-a7af-0fa8823b7e50 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.475286341598592, -3.816194199314987, -2.787960748187886, -2.0846149085976284
LBFGS Accumulation History: 3 points
Removed measurement 3d3f783 to history. Total: 3
th(0)=-6.475286341598592;dx=-1.234341387888206E-5
Armijo: th(406139.0229411184)=-5.21207495787414; dx=-3.30168609066348E-6 evalInputDelta=-1.2632113837244514
New Minimum: -6.475286341598592 > -6.670026029160994
END: th(203069.5114705592)=-6.670026029160994; dx=-6.813497717446966E-6 evalInputDelta=0.1947396875624019
Fitness changed from -6.475286341598592 to -6.670026029160994
Iteration 14 complete. Error: -6.670026029160994 Total: 2.6459; Orientation: 0.2924; Line Search: 1.7776
Adding measurement 5b352fd8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.580e+04, gradient 3.484e-03, dot -0.239; [b59dae98-ba75-4ec7-a7af-0fa8823b7e50 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.670026029160994, -3.816194199314987, -2.787960748187886, -2.0846149085976284
LBFGS Accumulation History: 3 points
Removed measurement 5b352fd8 to history. Total: 3
th(0)=-6.670026029160994;dx=-1.2090959377206409E-5
Armijo: th(437500.0000000003)=-5.038117299961563; dx=-2.7227180160007406E-6 evalInputDelta=-1.6319087291994308
New Minimum: -6.670026029160994 > -6.82002656846573
END: th(218750.00000000015)=-6.82002656846573; dx=-6.263595924108952E-6 evalInputDelta=0.15000053930473634
Fitness changed from -6.670026029160994 to -6.82002656846573
Iteration 15 complete. Error: -6.82002656846573 Total: 2.6811; Orientation: 0.2930; Line Search: 1.7993
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -6.82002656846573 (> -Infinity) after 44.528s (< 5400.000s)
Returns
-6.82002656846573