BasicOptimizer.scala:89 executed in 109.77 seconds (1.080 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 3762819482947
Reset training subject: 3763927387818
Adding measurement 292f21e to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.833358149374504;dx=-5.740740133888495E-5
New Minimum: -0.833358149374504 > -0.8334980625138955
WOLFE (weak): th(2.154434690031884)=-0.8334980625138955; dx=-5.740554310954304E-5 evalInputDelta=1.3991313939143435E-4
New Minimum: -0.8334980625138955 > -0.8336385078299721
WOLFE (weak): th(4.308869380063768)=-0.8336385078299721; dx=-5.73048348785089E-5 evalInputDelta=2.8035845546803273E-4
New Minimum: -0.8336385078299721 > -0.8341988310143704
WOLFE (weak): th(12.926608140191302)=-0.8341988310143704; dx=-5.743871706502024E-5 evalInputDelta=8.406816398663919E-4
New Minimum: -0.8341988310143704 > -0.8367205690281933
WOLFE (weak): th(51.70643256076521)=-0.8367205690281933; dx=-5.74127110809504E-5 evalInputDelta=0.0033624196536892192
New Minimum: -0.8367205690281933 > -0.8501201893714849
WOLFE (weak): th(258.53216280382605)=-0.8501201893714849; dx=-5.687012733171664E-5 evalInputDelta=0.016762039996980893
New Minimum: -0.8501201893714849 > -0.9319278180278411
WOLFE (weak): th(1551.1929768229563)=-0.9319278180278411; dx=-5.532486489576762E-5 evalInputDelta=0.09856966865333705
New Minimum: -0.9319278180278411 > -1.4005421632198505
END: th(10858.350837760694)=-1.4005421632198505; dx=-2.785161082411608E-5 evalInputDelta=0.5671840138453464
Fitness changed from -0.833358149374504 to -1.4005421632198505
Iteration 1 complete. Error: -1.4005421632198505 Total: 12.4190; Orientation: 0.0198; Line Search: 9.0958
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 708bac4 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-1.4005421632198505;dx=-7.035936165836249E-5
New Minimum: -1.4005421632198505 > -2.387457969098233
END: th(23393.607721408407)=-2.387457969098233; dx=-1.9012786237059345E-5 evalInputDelta=0.9869158058783827
Fitness changed from -1.4005421632198505 to -2.387457969098233
Iteration 2 complete. Error: -2.387457969098233 Total: 3.4220; Orientation: 0.0204; Line Search: 2.2890
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 2903399e to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-2.387457969098233;dx=-1.8090921779202434E-4
New Minimum: -2.387457969098233 > -4.446349763672031
END: th(50400.00000000001)=-4.446349763672031; dx=-1.9230814778970137E-5 evalInputDelta=2.058891794573798
Fitness changed from -2.387457969098233 to -4.446349763672031
Iteration 3 complete. Error: -4.446349763672031 Total: 3.3793; Orientation: 0.0196; Line Search: 2.2657
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 34b54435 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.390e+03, gradient 1.987e-02, dot -0.109; [ac74a7c7-e9de-4d63-a37e-19a1121d5d32 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.446349763672031, -2.387457969098233, -1.4005421632198505, -0.833358149374504
LBFGS Accumulation History: 3 points
Removed measurement 34b54435 to history. Total: 3
th(0)=-4.446349763672031;dx=-3.9461811540041563E-4
New Minimum: -4.446349763672031 > -8.407277457983984
END: th(108583.50837760697)=-8.407277457983984; dx=-1.3325211010060088E-5 evalInputDelta=3.9609276943119527
Fitness changed from -4.446349763672031 to -8.407277457983984
Iteration 4 complete. Error: -8.407277457983984 Total: 3.8909; Orientation: 0.4658; Line Search: 2.3071
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 4647b4f0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.877e+03, gradient 2.284e-02, dot -0.081; [ac74a7c7-e9de-4d63-a37e-19a1121d5d32 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.407277457983984, -2.387457969098233, -1.4005421632198505, -0.833358149374504
LBFGS Accumulation History: 3 points
Removed measurement 4647b4f0 to history. Total: 3
th(0)=-8.407277457983984;dx=-5.212494581451309E-4
New Minimum: -8.407277457983984 > -14.376224310435145
WOLF (strong): th(233936.07721408413)=-14.376224310435145; dx=1.0468676423480855E-6 evalInputDelta=5.968946852451161
New Minimum: -14.376224310435145 > -16.390110721515015
END: th(116968.03860704206)=-16.390110721515015; dx=-8.74094484357157E-6 evalInputDelta=7.982833263531031
Fitness changed from -8.407277457983984 to -16.390110721515015
Iteration 5 complete. Error: -16.390110721515015 Total: 5.0532; Orientation: 0.4695; Line Search: 3.4793
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 5600404 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.123e+04, gradient 3.398e-02, dot -0.060; [ac74a7c7-e9de-4d63-a37e-19a1121d5d32 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.390110721515015, -2.387457969098233, -1.4005421632198505, -0.833358149374504
LBFGS Accumulation History: 3 points
Removed measurement 5600404 to history. Total: 3
th(0)=-16.390110721515015;dx=-0.0011541439268124847
New Minimum: -16.390110721515015 > -19.07385457789562
WOLF (strong): th(252000.0000000001)=-19.07385457789562; dx=1.813452834607335E-5 evalInputDelta=2.6837438563806053
New Minimum: -19.07385457789562 > -27.764688060140525
WOLF (strong): th(126000.00000000004)=-27.764688060140525; dx=1.949324672431034E-5 evalInputDelta=11.37457733862551
New Minimum: -27.764688060140525 > -29.56632356042607
END: th(42000.000000000015)=-29.56632356042607; dx=-8.392167472822191E-5 evalInputDelta=13.176212838911056
Fitness changed from -16.390110721515015 to -29.56632356042607
Iteration 6 complete. Error: -29.56632356042607 Total: 6.2108; Orientation: 0.4724; Line Search: 4.6227
Adding measurement 54e03827 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.632e+05, gradient 3.886e-02, dot -0.058; [ac74a7c7-e9de-4d63-a37e-19a1121d5d32 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -29.56632356042607, -2.387457969098233, -1.4005421632198505, -0.833358149374504
LBFGS Accumulation History: 3 points
Removed measurement 54e03827 to history. Total: 3
th(0)=-29.56632356042607;dx=-0.0015099995669192025
New Minimum: -29.56632356042607 > -34.06865586448738
WOLF (strong): th(90486.25698133915)=-34.06865586448738; dx=4.094662365733118E-5 evalInputDelta=4.502332304061312
New Minimum: -34.06865586448738 > -39.342111700908866
WOLF (strong): th(45243.12849066957)=-39.342111700908866; dx=5.102064083029215E-5 evalInputDelta=9.775788140482796
New Minimum: -39.342111700908866 > -40.319935178141755
END: th(15081.04283022319)=-40.319935178141755; dx=-2.7851329274791855E-4 evalInputDelta=10.753611617715684
Fitness changed from -29.56632356042607 to -40.319935178141755
Iteration 7 complete. Error: -40.319935178141755 Total: 6.1392; Orientation: 0.4736; Line Search: 4.5814
Adding measurement 298c881e to history. Total: 3
Accepted: LBFGS Orientation magnitude: 3.436e+03, gradient 4.060e-02, dot 0.183; [ac74a7c7-e9de-4d63-a37e-19a1121d5d32 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-40.319935178141755;dx=-25.57823483029001
Armijo: th(32491.12183528946)=-9.408560320117163; dx=1.5184286135787762E-4 evalInputDelta=-30.911374858024594
Armijo: th(16245.56091764473)=-9.395587071185789; dx=3.09518907032077E-4 evalInputDelta=-30.924348106955968
Armijo: th(5415.186972548244)=-9.35883999598951; dx=8.705500420862368E-4 evalInputDelta=-30.961095182152246
Armijo: th(1353.796743137061)=-9.369012494698467; dx=0.0036551255189652136 evalInputDelta=-30.950922683443288
Armijo: th(270.7593486274122)=-9.09433991337739; dx=0.017923395499327407 evalInputDelta=-31.225595264764365
Armijo: th(45.126558104568694)=-8.821140606625399; dx=0.11091803811651502 evalInputDelta=-31.498794571516356
Armijo: th(6.446651157795528)=-27.72548303482582; dx=1.2513340343408634 evalInputDelta=-12.594452143315934
New Minimum: -40.319935178141755 > -48.1411955273786
END: th(0.805831394724441)=-48.1411955273786; dx=-6.029442917008538 evalInputDelta=7.821260349236844
Fitness changed from -40.319935178141755 to -48.1411955273786
Iteration 8 complete. Error: -48.1411955273786 Total: 13.2193; Orientation: 0.4659; Line Search: 11.6621
Adding measurement 26706407 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 5.344e+03, gradient 4.074e-02, dot -0.036; [ac74a7c7-e9de-4d63-a37e-19a1121d5d32 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -48.1411955273786, -40.319935178141755, -2.387457969098233, -1.4005421632198505, -0.833358149374504
Accepted: LBFGS Orientation magnitude: 2.237e+03, gradient 4.074e-02, dot 0.017; [ac74a7c7-e9de-4d63-a37e-19a1121d5d32 = 1.000/1.000e+00]
Overwriting history with 4 points
th(0)=-48.1411955273786;dx=-1.533835390358579
Armijo: th(1.7361111111111116)=-41.43079429578774; dx=1.5855920900043978 evalInputDelta=-6.710401231590858
Armijo: th(0.8680555555555558)=-45.02585663209828; dx=1.5983208049353987 evalInputDelta=-3.115338895280317
Armijo: th(0.2893518518518519)=-47.28312308078393; dx=1.3439461004996385 evalInputDelta=-0.8580724465946687
Armijo: th(0.07233796296296298)=-47.94887734135054; dx=0.9167181452482882 evalInputDelta=-0.19231818602806072
Armijo: th(0.014467592592592596)=-48.103605405853166; dx=1.0646005176517277 evalInputDelta=-0.03759012152543306
Armijo: th(0.002411265432098766)=-48.134975904562744; dx=0.9875965668783028 evalInputDelta=-0.00621962281585553
Armijo: th(3.444664902998237E-4)=-48.14030743439089; dx=1.0124573067517522 evalInputDelta=-8.88092987707978E-4
Armijo: th(4.3058311287477965E-5)=-48.14108517352006; dx=1.0181392364697053 evalInputDelta=-1.1035385853830348E-4
Armijo: th(4.784256809719774E-6)=-48.14118764496852; dx=1.018949474255949 evalInputDelta=-7.882410081094804E-6
New Minimum: -48.1411955273786 > -48.14119815484833
WOLF (strong): th(4.784256809719774E-7)=-48.14119815484833; dx=1.0193147710988044 evalInputDelta=2.627469733340604E-6
Armijo: th(4.349324372472522E-8)=-48.1411955273786; dx=1.0180860405013032 evalInputDelta=0.0
Armijo: th(3.624436977060435E-9)=-48.1411955273786; dx=1.01868131570926 evalInputDelta=0.0
WOLF (strong): th(2.7880284438926424E-10)=-48.14119815484833; dx=1.0220154671525927 evalInputDelta=2.627469733340604E-6
MIN ALPHA (1.9914488884947444E-11): th(4.784256809719774E-7)=-48.14119815484833
Fitness changed from -48.1411955273786 to -48.14119815484833
Iteration 9 complete. Error: -48.14119815484833 Total: 19.4808; Orientation: 1.0361; Line Search: 17.3481
Adding measurement 253575e3 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.434e+08, gradient 4.074e-02, dot -0.005; [ac74a7c7-e9de-4d63-a37e-19a1121d5d32 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -48.14119815484833, -48.1411955273786, -40.319935178141755, -2.387457969098233, -1.4005421632198505
Accepted: LBFGS Orientation magnitude: 1.985e+08, gradient 4.074e-02, dot 0.011; [ac74a7c7-e9de-4d63-a37e-19a1121d5d32 = 1.000/1.000e+00]
Overwriting history with 4 points
th(0)=-48.14119815484833;dx=-84726.71985712714
Armijo: th(2.1544346900318838E-10)=-48.14119027243869; dx=40043.70302052783 evalInputDelta=-7.882409640558308E-6
Armijo: th(1.0772173450159419E-10)=-48.14119027243869; dx=40217.82341942568 evalInputDelta=-7.882409640558308E-6
MIN ALPHA (3.590724483386473E-11): th(0.0)=-48.14119815484833
Fitness changed from -48.14119815484833 to -48.14119815484833
Static Iteration Total: 6.6970; Orientation: 1.0246; Line Search: 4.5842
Iteration 10 failed. Error: -48.14119815484833
Previous Error: 0.0 -> -48.14119815484833
Retrying iteration 10
Reset training subject: 3842731236114
Adding measurement 51efb9da to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-48.14119815484833;dx=-0.001659195447210385
WOLFE (weak): th(2.1544346900318838E-10)=-48.14119815484833; dx=-0.0016178658352212735 evalInputDelta=0.0
WOLFE (weak): th(4.3088693800637676E-10)=-48.14119815484833; dx=-0.001617945685828939 evalInputDelta=0.0
WOLFE (weak): th(1.2926608140191302E-9)=-48.14119815484833; dx=-0.0016179980498579266 evalInputDelta=0.0
WOLFE (weak): th(5.170643256076521E-9)=-48.14119815484833; dx=-0.0016179115005461784 evalInputDelta=0.0
Armijo: th(2.5853216280382606E-8)=-48.1411955273786; dx=-0.0016179425789339648 evalInputDelta=-2.627469733340604E-6
WOLFE (weak): th(1.5511929768229562E-8)=-48.14119815484833; dx=-0.0016178861382134297 evalInputDelta=0.0
WOLFE (weak): th(2.0682573024306084E-8)=-48.14119815484833; dx=-0.0016179895436772985 evalInputDelta=0.0
WOLFE (weak): th(2.3267894652344343E-8)=-48.14119815484833; dx=-0.0016178905783672476 evalInputDelta=0.0
Armijo: th(2.4560555466363474E-8)=-48.1411955273786; dx=-0.0016179772220763062 evalInputDelta=-2.627469733340604E-6
Armijo: th(2.391422505935391E-8)=-48.1411955273786; dx=-0.0016179225385967394 evalInputDelta=-2.627469733340604E-6
WOLFE (weak): th(2.3591059855849126E-8)=-48.14119815484833; dx=-0.001617895029298958 evalInputDelta=0.0
Armijo: th(2.3752642457601516E-8)=-48.1411955273786; dx=-0.0016179498366574428 evalInputDelta=-2.627469733340604E-6
Armijo: th(2.367185115672532E-8)=-48.1411955273786; dx=-0.0016179450314005933 evalInputDelta=-2.627469733340604E-6
Armijo: th(2.363145550628722E-8)=-48.1411955273786; dx=-0.0016179091595503114 evalInputDelta=-2.627469733340604E-6
Armijo: th(2.3611257681068175E-8)=-48.1411955273786; dx=-0.0016178161309168314 evalInputDelta=-2.627469733340604E-6
Armijo: th(2.360115876845865E-8)=-48.1411955273786; dx=-0.0016178996567575968 evalInputDelta=-2.627469733340604E-6
WOLFE (weak): th(2.359610931215389E-8)=-48.14119815484833; dx=-0.0016178818650015264 evalInputDelta=0.0
Armijo: th(2.359863404030627E-8)=-48.1411955273786; dx=-0.0016178716881752359 evalInputDelta=-2.627469733340604E-6
WOLFE (weak): th(2.359737167623008E-8)=-48.14119815484833; dx=-0.0016179507798426795 evalInputDelta=0.0
Armijo: th(2.3598002858268175E-8)=-48.1411955273786; dx=-0.0016179114199629768 evalInputDelta=-2.627469733340604E-6
WOLFE (weak): th(2.3597687267249126E-8)=-48.14119815484833; dx=-0.0016178989395621759 evalInputDelta=0.0
Armijo: th(2.359784506275865E-8)=-48.1411955273786; dx=-0.0016179929015126809 evalInputDelta=-2.627469733340604E-6
mu ~= nu (2.3597687267249126E-8): th(0.0)=-48.14119815484833
Fitness changed from -48.14119815484833 to -48.14119815484833
Static Iteration Total: 29.8578; Orientation: 0.0193; Line Search: 27.6500
Iteration 11 failed. Error: -48.14119815484833
Previous Error: 0.0 -> -48.14119815484833
Optimization terminated 11
Final threshold in iteration 11: -48.14119815484833 (> -Infinity) after 109.769s (< 3600.000s)
Returns
-48.14119815484833