BasicOptimizer.scala:89 executed in 8.62 seconds (0.326 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 156015765892900
Reset training subject: 156015898056900
Adding measurement 352c1383 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-8.178134033945001;dx=-0.001362268000208765
New Minimum: -8.178134033945001 > -8.180905388157571
WOLFE (weak): th(2.154434690031884)=-8.180905388157571; dx=-0.001348619334792031 evalInputDelta=0.0027713542125695767
New Minimum: -8.180905388157571 > -8.183676258851884
WOLFE (weak): th(4.308869380063768)=-8.183676258851884; dx=-0.0013864013370163113 evalInputDelta=0.005542224906882609
New Minimum: -8.183676258851884 > -8.194773091016925
WOLFE (weak): th(12.926608140191302)=-8.194773091016925; dx=-0.0013668061178107574 evalInputDelta=0.016639057071923702
New Minimum: -8.194773091016925 > -8.24438129034896
WOLFE (weak): th(51.70643256076521)=-8.24438129034896; dx=-0.0013433318038482017 evalInputDelta=0.06624725640395823
New Minimum: -8.24438129034896 > -8.503915280270586
WOLFE (weak): th(258.53216280382605)=-8.503915280270586; dx=-0.0012994211216585554 evalInputDelta=0.3257812463255849
New Minimum: -8.503915280270586 > -10.26145338060312
END: th(1551.1929768229563)=-10.26145338060312; dx=-0.0012102208305760764 evalInputDelta=2.0833193466581186
Fitness changed from -8.178134033945001 to -10.26145338060312
Iteration 1 complete. Error: -10.26145338060312 Total: 1.1571; Orientation: 0.0018; Line Search: 0.8029
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 2609b00d to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-10.26145338060312;dx=-0.001727451465552821
New Minimum: -10.26145338060312 > -14.978253844281681
END: th(3341.943960201201)=-14.978253844281681; dx=-7.709819132020923E-4 evalInputDelta=4.716800463678561
Fitness changed from -10.26145338060312 to -14.978253844281681
Iteration 2 complete. Error: -14.978253844281681 Total: 0.3596; Orientation: 0.0015; Line Search: 0.2381
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 2421eea0 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-14.978253844281681;dx=-0.002089911131329329
New Minimum: -14.978253844281681 > -22.253932431853354
END: th(7200.000000000001)=-22.253932431853354; dx=-3.6998214308879064E-4 evalInputDelta=7.275678587571672
Fitness changed from -14.978253844281681 to -22.253932431853354
Iteration 3 complete. Error: -22.253932431853354 Total: 0.3695; Orientation: 0.0014; Line Search: 0.2425
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement f914915 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.130e+03, gradient 4.465e-02, dot -0.213; [c895377c-6d06-4647-ac1f-34e797fd1ebb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -22.253932431853354, -14.978253844281681, -10.26145338060312, -8.178134033945001
LBFGS Accumulation History: 3 points
Removed measurement f914915 to history. Total: 3
th(0)=-22.253932431853354;dx=-0.0019111570361711693
New Minimum: -22.253932431853354 > -30.162325411842964
END: th(15511.929768229566)=-30.162325411842964; dx=-1.825219380350923E-4 evalInputDelta=7.90839297998961
Fitness changed from -22.253932431853354 to -30.162325411842964
Iteration 4 complete. Error: -30.162325411842964 Total: 0.4029; Orientation: 0.0336; Line Search: 0.2344
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 27891d0d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.646e+03, gradient 4.879e-02, dot -0.125; [c895377c-6d06-4647-ac1f-34e797fd1ebb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -30.162325411842964, -14.978253844281681, -10.26145338060312, -8.178134033945001
LBFGS Accumulation History: 3 points
Removed measurement 27891d0d to history. Total: 3
th(0)=-30.162325411842964;dx=-0.0023111440543926226
New Minimum: -30.162325411842964 > -55.47380843211529
WOLF (strong): th(33419.43960201202)=-55.47380843211529; dx=9.149534804391634E-5 evalInputDelta=25.311483020272327
END: th(16709.71980100601)=-52.36370139703752; dx=-2.894675895044295E-4 evalInputDelta=22.201375985194552
Fitness changed from -30.162325411842964 to -55.47380843211529
Iteration 5 complete. Error: -55.47380843211529 Total: 0.5306; Orientation: 0.0288; Line Search: 0.3548
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 5036ce3d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.951e+02, gradient 5.388e-02, dot -0.049; [c895377c-6d06-4647-ac1f-34e797fd1ebb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -55.47380843211529, -14.978253844281681, -10.26145338060312, -8.178134033945001
LBFGS Accumulation History: 3 points
Removed measurement 5036ce3d to history. Total: 3
th(0)=-55.47380843211529;dx=-0.0028674266050416485
New Minimum: -55.47380843211529 > -98.03977423926544
WOLF (strong): th(36000.000000000015)=-98.03977423926544; dx=3.1871081669073136E-4 evalInputDelta=42.56596580715015
END: th(18000.000000000007)=-93.2770865786715; dx=-2.4388903234894195E-4 evalInputDelta=37.80327814655621
Fitness changed from -55.47380843211529 to -98.03977423926544
Iteration 6 complete. Error: -98.03977423926544 Total: 0.8794; Orientation: 0.0280; Line Search: 0.7299
Adding measurement d9ec78d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.059e+03, gradient 5.707e-02, dot -0.130; [c895377c-6d06-4647-ac1f-34e797fd1ebb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -98.03977423926544, -14.978253844281681, -10.26145338060312, -8.178134033945001
LBFGS Accumulation History: 3 points
Removed measurement d9ec78d to history. Total: 3
th(0)=-98.03977423926544;dx=-0.003208800012224295
New Minimum: -98.03977423926544 > -136.9488410933112
WOLF (strong): th(38779.824420573925)=-136.9488410933112; dx=6.374539429914902E-4 evalInputDelta=38.90906685404575
New Minimum: -136.9488410933112 > -146.6049830807569
END: th(19389.912210286962)=-146.6049830807569; dx=-0.0010992582613064718 evalInputDelta=48.56520884149147
Fitness changed from -98.03977423926544 to -146.6049830807569
Iteration 7 complete. Error: -146.6049830807569 Total: 0.5121; Orientation: 0.0317; Line Search: 0.3626
Adding measurement 1cfff096 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.816e+05, gradient 5.601e-02, dot -0.168; [c895377c-6d06-4647-ac1f-34e797fd1ebb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -146.6049830807569, -14.978253844281681, -10.26145338060312, -8.178134033945001
LBFGS Accumulation History: 3 points
Removed measurement 1cfff096 to history. Total: 3
th(0)=-146.6049830807569;dx=-0.003052236602667524
New Minimum: -146.6049830807569 > -174.44019053801802
WOLF (strong): th(41774.29950251503)=-174.44019053801802; dx=2.515009651663123E-4 evalInputDelta=27.835207457261106
New Minimum: -174.44019053801802 > -183.0940873241312
END: th(20887.149751257515)=-183.0940873241312; dx=-4.4320272964595603E-4 evalInputDelta=36.4891042433743
Fitness changed from -146.6049830807569 to -183.0940873241312
Iteration 8 complete. Error: -183.0940873241312 Total: 0.5095; Orientation: 0.0288; Line Search: 0.3717
Adding measurement 2c434a27 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 3.492e+04, gradient 5.350e-02, dot 0.190; [c895377c-6d06-4647-ac1f-34e797fd1ebb = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-183.0940873241312;dx=-338.5484190762472
Armijo: th(45000.00000000002)=-76.13045165509601; dx=3.6532022042470836E-4 evalInputDelta=-106.9636356690352
Armijo: th(22500.00000000001)=-76.13045165509601; dx=7.714629421408586E-4 evalInputDelta=-106.9636356690352
Armijo: th(7500.000000000004)=-76.13045165509601; dx=0.0024347501365508187 evalInputDelta=-106.9636356690352
Armijo: th(1875.000000000001)=-76.3354922316507; dx=0.009936602917773811 evalInputDelta=-106.7585950924805
Armijo: th(375.00000000000017)=-76.25466781777185; dx=0.04289780849680513 evalInputDelta=-106.83941950635936
Armijo: th(62.50000000000003)=-76.25515220095429; dx=0.29601420882867424 evalInputDelta=-106.83893512317692
Armijo: th(8.928571428571432)=-76.42509641354238; dx=2.3711322396851866 evalInputDelta=-106.66899091058883
Armijo: th(1.116071428571429)=-83.28831787934006; dx=35.24028394462348 evalInputDelta=-99.80576944479115
Armijo: th(0.12400793650793657)=-170.78263105141494; dx=105.2990093276108 evalInputDelta=-12.31145627271627
New Minimum: -183.0940873241312 > -186.78135741778496
END: th(0.012400793650793657)=-186.78135741778496; dx=-60.78163015408474 evalInputDelta=3.6872700936537512
Fitness changed from -183.0940873241312 to -186.78135741778496
Iteration 9 complete. Error: -186.78135741778496 Total: 1.5631; Orientation: 0.0379; Line Search: 1.4029
Adding measurement 1f4b1412 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.045e+05, gradient 5.443e-02, dot -0.145; [c895377c-6d06-4647-ac1f-34e797fd1ebb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -186.78135741778496, -183.0940873241312, -14.978253844281681, -10.26145338060312, -8.178134033945001
Rejected: LBFGS Orientation magnitude: 1.431e+05, gradient 5.443e-02, dot -0.246; [c895377c-6d06-4647-ac1f-34e797fd1ebb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -186.78135741778496, -183.0940873241312, -14.978253844281681, -10.26145338060312
LBFGS Accumulation History: 3 points
Removed measurement 1f4b1412 to history. Total: 4
Removed measurement 2c434a27 to history. Total: 3
th(0)=-186.78135741778496;dx=-0.0029368574885013338
New Minimum: -186.78135741778496 > -186.78142886405942
END: th(0.026716700025196986)=-186.78142886405942; dx=-0.002482626881461512 evalInputDelta=7.144627446109553E-5
Fitness changed from -186.78135741778496 to -186.78142886405942
Iteration 10 complete. Error: -186.78142886405942 Total: 0.4050; Orientation: 0.0632; Line Search: 0.2286
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 427c1acc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.382e+05, gradient 5.447e-02, dot -0.141; [c895377c-6d06-4647-ac1f-34e797fd1ebb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -186.78142886405942, -14.978253844281681, -10.26145338060312, -8.178134033945001
LBFGS Accumulation History: 3 points
Removed measurement 427c1acc to history. Total: 3
th(0)=-186.78142886405942;dx=-0.0028937129218536126
New Minimum: -186.78142886405942 > -186.78158196312697
END: th(0.05755938533746009)=-186.78158196312697; dx=-0.002453906418664837 evalInputDelta=1.5309906754623626E-4
Fitness changed from -186.78142886405942 to -186.78158196312697
Iteration 11 complete. Error: -186.78158196312697 Total: 0.3756; Orientation: 0.0271; Line Search: 0.2274
Adding measurement 85d41bc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.161e+06, gradient 5.516e-02, dot -0.142; [c895377c-6d06-4647-ac1f-34e797fd1ebb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -186.78158196312697, -14.978253844281681, -10.26145338060312, -8.178134033945001
LBFGS Accumulation History: 3 points
Removed measurement 85d41bc to history. Total: 3
th(0)=-186.78158196312697;dx=-0.002931375212994788
New Minimum: -186.78158196312697 > -186.7818983674689
END: th(0.1240079365079366)=-186.7818983674689; dx=-0.002466224840691993 evalInputDelta=3.1640434193036526E-4
Fitness changed from -186.78158196312697 to -186.7818983674689
Iteration 12 complete. Error: -186.7818983674689 Total: 0.3829; Orientation: 0.0303; Line Search: 0.2438
Adding measurement 768d819c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.170e+05, gradient 5.415e-02, dot -0.138; [c895377c-6d06-4647-ac1f-34e797fd1ebb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -186.7818983674689, -14.978253844281681, -10.26145338060312, -8.178134033945001
LBFGS Accumulation History: 3 points
Removed measurement 768d819c to history. Total: 3
th(0)=-186.7818983674689;dx=-0.0028972291614759115
New Minimum: -186.7818983674689 > -186.78260262037026
END: th(0.26716700025196993)=-186.78260262037026; dx=-0.0024388890889786912 evalInputDelta=7.042529013574494E-4
Fitness changed from -186.7818983674689 to -186.78260262037026
Iteration 13 complete. Error: -186.78260262037026 Total: 0.3915; Orientation: 0.0295; Line Search: 0.2485
Adding measurement 2ee95e95 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.778e+05, gradient 5.368e-02, dot -0.142; [c895377c-6d06-4647-ac1f-34e797fd1ebb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -186.78260262037026, -14.978253844281681, -10.26145338060312, -8.178134033945001
LBFGS Accumulation History: 3 points
Removed measurement 2ee95e95 to history. Total: 3
th(0)=-186.78260262037026;dx=-0.0028429553017751486
New Minimum: -186.78260262037026 > -186.78407235700251
END: th(0.5755938533746011)=-186.78407235700251; dx=-0.002423149380951039 evalInputDelta=0.0014697366322593552
Fitness changed from -186.78260262037026 to -186.78407235700251
Iteration 14 complete. Error: -186.78407235700251 Total: 0.3872; Orientation: 0.0338; Line Search: 0.2388
Adding measurement daa2c48 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.816e+05, gradient 5.417e-02, dot -0.140; [c895377c-6d06-4647-ac1f-34e797fd1ebb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -186.78407235700251, -14.978253844281681, -10.26145338060312, -8.178134033945001
LBFGS Accumulation History: 3 points
Removed measurement daa2c48 to history. Total: 3
th(0)=-186.78407235700251;dx=-0.002911722662056371
New Minimum: -186.78407235700251 > -186.78723633411764
END: th(1.2400793650793664)=-186.78723633411764; dx=-0.0024084139205463127 evalInputDelta=0.003163977115121952
Fitness changed from -186.78407235700251 to -186.78723633411764
Iteration 15 complete. Error: -186.78723633411764 Total: 0.3872; Orientation: 0.0319; Line Search: 0.2383
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -186.78723633411764 (> -Infinity) after 8.624s (< 5400.000s)
Returns
-186.78723633411764