BasicOptimizer.scala:89 executed in 11.23 seconds (0.389 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 636956953390900
Reset training subject: 636957147813000
Adding measurement 73ba1fc4 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-1.5712612982369554;dx=-1.0942015595418628E-5
New Minimum: -1.5712612982369554 > -1.5713158075810154
WOLFE (weak): th(2.154434690031884)=-1.5713158075810154; dx=-1.0956482817269172E-5 evalInputDelta=5.45093440600386E-5
New Minimum: -1.5713158075810154 > -1.571369874755398
WOLFE (weak): th(4.308869380063768)=-1.571369874755398; dx=-1.096927161912432E-5 evalInputDelta=1.0857651844253802E-4
New Minimum: -1.571369874755398 > -1.571587005288546
WOLFE (weak): th(12.926608140191302)=-1.571587005288546; dx=-1.0961388307299471E-5 evalInputDelta=3.2570705159051805E-4
New Minimum: -1.571587005288546 > -1.5725640736392705
WOLFE (weak): th(51.70643256076521)=-1.5725640736392705; dx=-1.0967679580309713E-5 evalInputDelta=0.0013027754023151594
New Minimum: -1.5725640736392705 > -1.57778748278164
WOLFE (weak): th(258.53216280382605)=-1.57778748278164; dx=-1.1037643157131498E-5 evalInputDelta=0.006526184544684677
New Minimum: -1.57778748278164 > -1.610758430886416
WOLFE (weak): th(1551.1929768229563)=-1.610758430886416; dx=-1.1263107459656767E-5 evalInputDelta=0.03949713264946064
New Minimum: -1.610758430886416 > -1.8622635879529263
WOLFE (weak): th(10858.350837760694)=-1.8622635879529263; dx=-1.2636539042104876E-5 evalInputDelta=0.291002289715971
New Minimum: -1.8622635879529263 > -3.7291927389188624
END: th(86866.80670208555)=-3.7291927389188624; dx=-6.352651485662037E-6 evalInputDelta=2.157931440681907
Fitness changed from -1.5712612982369554 to -3.7291927389188624
Iteration 1 complete. Error: -3.7291927389188624 Total: 1.8876; Orientation: 0.0021; Line Search: 1.3766
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 3a24f1fc to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-3.7291927389188624;dx=-1.855239285673745E-5
New Minimum: -3.7291927389188624 > -6.045005267271489
WOLF (strong): th(187148.86177126726)=-6.045005267271489; dx=1.973469638765593E-7 evalInputDelta=2.3158125283526267
END: th(93574.43088563363)=-5.791831356874437; dx=-3.210451475758172E-6 evalInputDelta=2.0626386179555745
Fitness changed from -3.7291927389188624 to -6.045005267271489
Iteration 2 complete. Error: -6.045005267271489 Total: 0.6270; Orientation: 0.0034; Line Search: 0.4562
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 726e9971 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-6.045005267271489;dx=-1.9495493891794658E-5
New Minimum: -6.045005267271489 > -8.282347762441582
WOLF (strong): th(201600.00000000003)=-8.282347762441582; dx=4.165065201971147E-6 evalInputDelta=2.2373424951700924
New Minimum: -8.282347762441582 > -8.732919739559906
END: th(100800.00000000001)=-8.732919739559906; dx=-2.4827317753986956E-6 evalInputDelta=2.6879144722884165
Fitness changed from -6.045005267271489 to -8.732919739559906
Iteration 3 complete. Error: -8.732919739559906 Total: 0.6315; Orientation: 0.0029; Line Search: 0.4588
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 3552f9d8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.883e+03, gradient 3.346e-03, dot -0.415; [a54f3f8a-d9b9-4de8-9f7e-1cc978caf58a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.732919739559906, -6.045005267271489, -3.7291927389188624, -1.5712612982369554
LBFGS Accumulation History: 3 points
Removed measurement 3552f9d8 to history. Total: 3
th(0)=-8.732919739559906;dx=-1.1130491814079808E-5
New Minimum: -8.732919739559906 > -10.2442046727478
WOLF (strong): th(217167.01675521393)=-10.2442046727478; dx=5.131376232378085E-7 evalInputDelta=1.5112849331878948
New Minimum: -10.2442046727478 > -10.262876551446015
END: th(108583.50837760697)=-10.262876551446015; dx=-2.2215774558320965E-6 evalInputDelta=1.5299568118861089
Fitness changed from -8.732919739559906 to -10.262876551446015
Iteration 4 complete. Error: -10.262876551446015 Total: 0.6719; Orientation: 0.0397; Line Search: 0.4620
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 92e0192 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.796e+03, gradient 3.247e-03, dot -0.429; [a54f3f8a-d9b9-4de8-9f7e-1cc978caf58a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.262876551446015, -6.045005267271489, -3.7291927389188624, -1.5712612982369554
LBFGS Accumulation History: 3 points
Removed measurement 92e0192 to history. Total: 3
th(0)=-10.262876551446015;dx=-1.0509500312453495E-5
New Minimum: -10.262876551446015 > -11.646495619157816
WOLF (strong): th(233936.07721408413)=-11.646495619157816; dx=6.866870151902414E-7 evalInputDelta=1.3836190677118019
New Minimum: -11.646495619157816 > -11.73893487256443
END: th(116968.03860704206)=-11.73893487256443; dx=-1.6946124842099738E-6 evalInputDelta=1.4760583211184155
Fitness changed from -10.262876551446015 to -11.73893487256443
Iteration 5 complete. Error: -11.73893487256443 Total: 0.6682; Orientation: 0.0395; Line Search: 0.4603
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 43751ca6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.935e+03, gradient 3.243e-03, dot -0.470; [a54f3f8a-d9b9-4de8-9f7e-1cc978caf58a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.73893487256443, -6.045005267271489, -3.7291927389188624, -1.5712612982369554
LBFGS Accumulation History: 3 points
Removed measurement 43751ca6 to history. Total: 3
th(0)=-11.73893487256443;dx=-1.0497777790596936E-5
New Minimum: -11.73893487256443 > -13.174798849242494
WOLF (strong): th(252000.0000000001)=-13.174798849242494; dx=6.26403758258495E-7 evalInputDelta=1.435863976678064
New Minimum: -13.174798849242494 > -13.328835797379615
END: th(126000.00000000004)=-13.328835797379615; dx=-1.6090851118462504E-6 evalInputDelta=1.5899009248151845
Fitness changed from -11.73893487256443 to -13.328835797379615
Iteration 6 complete. Error: -13.328835797379615 Total: 0.6705; Orientation: 0.0397; Line Search: 0.4661
Adding measurement addcd9d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.779e+03, gradient 3.243e-03, dot -0.471; [a54f3f8a-d9b9-4de8-9f7e-1cc978caf58a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -13.328835797379615, -6.045005267271489, -3.7291927389188624, -1.5712612982369554
LBFGS Accumulation History: 3 points
Removed measurement addcd9d to history. Total: 3
th(0)=-13.328835797379615;dx=-1.0502604352275584E-5
New Minimum: -13.328835797379615 > -14.966803057185304
WOLF (strong): th(271458.77094401745)=-14.966803057185304; dx=5.603178429054457E-7 evalInputDelta=1.6379672598056896
New Minimum: -14.966803057185304 > -15.053909415069084
END: th(135729.38547200873)=-15.053909415069084; dx=-1.7729385562742283E-6 evalInputDelta=1.7250736176894694
Fitness changed from -13.328835797379615 to -15.053909415069084
Iteration 7 complete. Error: -15.053909415069084 Total: 0.6399; Orientation: 0.0389; Line Search: 0.4504
Adding measurement 6ef73fdd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.643e+03, gradient 3.192e-03, dot -0.527; [a54f3f8a-d9b9-4de8-9f7e-1cc978caf58a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -15.053909415069084, -6.045005267271489, -3.7291927389188624, -1.5712612982369554
LBFGS Accumulation History: 3 points
Removed measurement 6ef73fdd to history. Total: 3
th(0)=-15.053909415069084;dx=-1.0169422010394244E-5
New Minimum: -15.053909415069084 > -16.94215247249162
WOLF (strong): th(292420.0965176052)=-16.94215247249162; dx=4.779968812793311E-7 evalInputDelta=1.888243057422537
New Minimum: -16.94215247249162 > -16.943277795629307
END: th(146210.0482588026)=-16.943277795629307; dx=-2.0322963285456402E-6 evalInputDelta=1.8893683805602226
Fitness changed from -15.053909415069084 to -16.943277795629307
Iteration 8 complete. Error: -16.943277795629307 Total: 0.6445; Orientation: 0.0407; Line Search: 0.4482
Adding measurement 1ad351ac to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.807e+04, gradient 3.175e-03, dot -0.528; [a54f3f8a-d9b9-4de8-9f7e-1cc978caf58a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.943277795629307, -6.045005267271489, -3.7291927389188624, -1.5712612982369554
LBFGS Accumulation History: 3 points
Removed measurement 1ad351ac to history. Total: 3
th(0)=-16.943277795629307;dx=-1.0031529450655321E-5
New Minimum: -16.943277795629307 > -18.959702872212837
WOLF (strong): th(315000.0000000001)=-18.959702872212837; dx=3.9323726819561314E-7 evalInputDelta=2.0164250765835305
END: th(157500.00000000006)=-18.921190961046083; dx=-2.087128831438954E-6 evalInputDelta=1.9779131654167763
Fitness changed from -16.943277795629307 to -18.959702872212837
Iteration 9 complete. Error: -18.959702872212837 Total: 0.6330; Orientation: 0.0383; Line Search: 0.4485
Adding measurement 7e9ee2dd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.531e+04, gradient 4.331e-03, dot -0.397; [a54f3f8a-d9b9-4de8-9f7e-1cc978caf58a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.959702872212837, -6.045005267271489, -3.7291927389188624, -1.5712612982369554
LBFGS Accumulation History: 3 points
Removed measurement 7e9ee2dd to history. Total: 3
th(0)=-18.959702872212837;dx=-1.8696855040847385E-5
New Minimum: -18.959702872212837 > -21.694182583293713
WOLF (strong): th(339323.4636800218)=-21.694182583293713; dx=5.378357603150785E-6 evalInputDelta=2.7344797110808763
New Minimum: -21.694182583293713 > -23.08235431408195
END: th(169661.7318400109)=-23.08235431408195; dx=-2.0586688782379615E-6 evalInputDelta=4.122651441869113
Fitness changed from -18.959702872212837 to -23.08235431408195
Iteration 10 complete. Error: -23.08235431408195 Total: 1.0724; Orientation: 0.0389; Line Search: 0.8876
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 4770c0ac to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.313e+04, gradient 3.003e-03, dot -0.584; [a54f3f8a-d9b9-4de8-9f7e-1cc978caf58a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -23.08235431408195, -6.045005267271489, -3.7291927389188624, -1.5712612982369554
LBFGS Accumulation History: 3 points
Removed measurement 4770c0ac to history. Total: 3
th(0)=-23.08235431408195;dx=-8.99331359566396E-6
New Minimum: -23.08235431408195 > -26.46038401215143
END: th(365525.1206470065)=-26.46038401215143; dx=-1.2724109196035756E-6 evalInputDelta=3.3780296980694793
Fitness changed from -23.08235431408195 to -26.46038401215143
Iteration 11 complete. Error: -26.46038401215143 Total: 0.5088; Orientation: 0.0393; Line Search: 0.3028
Adding measurement 45d66da9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.955e+04, gradient 3.960e-03, dot -0.452; [a54f3f8a-d9b9-4de8-9f7e-1cc978caf58a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -26.46038401215143, -6.045005267271489, -3.7291927389188624, -1.5712612982369554
LBFGS Accumulation History: 3 points
Removed measurement 45d66da9 to history. Total: 3
th(0)=-26.46038401215143;dx=-1.565053998664291E-5
Armijo: th(787500.0000000003)=-25.356605590534947; dx=3.85495521297382E-6 evalInputDelta=-1.1037784216164823
New Minimum: -26.46038401215143 > -29.28260967858821
WOLF (strong): th(393750.0000000002)=-29.28260967858821; dx=3.6819258497588538E-6 evalInputDelta=2.822225666436779
New Minimum: -29.28260967858821 > -29.567148172156045
END: th(131250.00000000006)=-29.567148172156045; dx=-6.149242228794309E-6 evalInputDelta=3.106764160004616
Fitness changed from -26.46038401215143 to -29.567148172156045
Iteration 12 complete. Error: -29.567148172156045 Total: 0.7949; Orientation: 0.0408; Line Search: 0.6025
Adding measurement 5dfe0b77 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.053e+05, gradient 2.611e-03, dot -0.700; [a54f3f8a-d9b9-4de8-9f7e-1cc978caf58a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -29.567148172156045, -6.045005267271489, -3.7291927389188624, -1.5712612982369554
LBFGS Accumulation History: 3 points
Removed measurement 5dfe0b77 to history. Total: 3
th(0)=-29.567148172156045;dx=-6.791730013354974E-6
New Minimum: -29.567148172156045 > -32.88985725343737
END: th(282769.55306668486)=-32.88985725343737; dx=-4.729270331465723E-6 evalInputDelta=3.3227090812813245
Fitness changed from -29.567148172156045 to -32.88985725343737
Iteration 13 complete. Error: -32.88985725343737 Total: 0.4853; Orientation: 0.0391; Line Search: 0.3002
Adding measurement 5ed9defe to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.378e+05, gradient 2.644e-03, dot -0.653; [a54f3f8a-d9b9-4de8-9f7e-1cc978caf58a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -32.88985725343737, -6.045005267271489, -3.7291927389188624, -1.5712612982369554
LBFGS Accumulation History: 3 points
Removed measurement 5ed9defe to history. Total: 3
th(0)=-32.88985725343737;dx=-6.972422760878319E-6
New Minimum: -32.88985725343737 > -37.685559244048896
END: th(609208.5344116775)=-37.685559244048896; dx=-2.2635614808230753E-6 evalInputDelta=4.795701990611526
Fitness changed from -32.88985725343737 to -37.685559244048896
Iteration 14 complete. Error: -37.685559244048896 Total: 0.4836; Orientation: 0.0385; Line Search: 0.3010
Adding measurement 684c4902 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.372e+04, gradient 3.452e-03, dot -0.456; [a54f3f8a-d9b9-4de8-9f7e-1cc978caf58a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -37.685559244048896, -6.045005267271489, -3.7291927389188624, -1.5712612982369554
LBFGS Accumulation History: 3 points
Removed measurement 684c4902 to history. Total: 3
th(0)=-37.685559244048896;dx=-1.1884069271767628E-5
Armijo: th(1312500.0000000007)=-35.21871728465369; dx=3.502674060510123E-6 evalInputDelta=-2.466841959395204
New Minimum: -37.685559244048896 > -40.91014676756117
WOLF (strong): th(656250.0000000003)=-40.91014676756117; dx=3.207473191151192E-6 evalInputDelta=3.2245875235122767
New Minimum: -40.91014676756117 > -41.58348779031533
END: th(218750.00000000012)=-41.58348779031533; dx=-4.236626688930874E-6 evalInputDelta=3.897928546266435
Fitness changed from -37.685559244048896 to -41.58348779031533
Iteration 15 complete. Error: -41.58348779031533 Total: 0.7904; Orientation: 0.0398; Line Search: 0.6027
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -41.58348779031533 (> -Infinity) after 11.225s (< 5400.000s)
Returns
-41.58348779031533