BasicOptimizer.scala:89 executed in 55.17 seconds (1.634 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 210949762383100
Reset training subject: 210950580777700
Adding measurement 7f25ba11 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=9.856489925517568;dx=-2.4691695228177922E-5
New Minimum: 9.856489925517568 > 9.856451291158542
WOLFE (weak): th(2.154434690031884)=9.856451291158542; dx=-2.4717355060949057E-5 evalInputDelta=3.863435902573542E-5
New Minimum: 9.856451291158542 > 9.856416381021091
WOLFE (weak): th(4.308869380063768)=9.856416381021091; dx=-2.470613293501515E-5 evalInputDelta=7.354449647678507E-5
New Minimum: 9.856416381021091 > 9.8562698402702
WOLFE (weak): th(12.926608140191302)=9.8562698402702; dx=-2.469623647968729E-5 evalInputDelta=2.2008524736882862E-4
New Minimum: 9.8562698402702 > 9.855612428299946
WOLFE (weak): th(51.70643256076521)=9.855612428299946; dx=-2.4698620405755503E-5 evalInputDelta=8.77497217622647E-4
New Minimum: 9.855612428299946 > 9.852100136870884
WOLFE (weak): th(258.53216280382605)=9.852100136870884; dx=-2.4673862992512893E-5 evalInputDelta=0.004389788646683712
New Minimum: 9.852100136870884 > 9.830193820884723
WOLFE (weak): th(1551.1929768229563)=9.830193820884723; dx=-2.455334060357989E-5 evalInputDelta=0.02629610463284493
New Minimum: 9.830193820884723 > 9.676905809263246
WOLFE (weak): th(10858.350837760694)=9.676905809263246; dx=-2.3079149551023987E-5 evalInputDelta=0.1795841162543219
New Minimum: 9.676905809263246 > 8.894850295420973
END: th(86866.80670208555)=8.894850295420973; dx=-9.599914464180836E-6 evalInputDelta=0.9616396300965953
Fitness changed from 9.856489925517568 to 8.894850295420973
Iteration 1 complete. Error: 8.894850295420973 Total: 9.4868; Orientation: 0.0223; Line Search: 7.1090
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 68b7baaf to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=8.894850295420973;dx=-2.093130330681865E-5
New Minimum: 8.894850295420973 > 7.8039398737250725
END: th(187148.86177126726)=7.8039398737250725; dx=-5.633200112310686E-6 evalInputDelta=1.0909104216959005
Fitness changed from 8.894850295420973 to 7.8039398737250725
Iteration 2 complete. Error: 7.8039398737250725 Total: 2.1812; Orientation: 0.0160; Line Search: 1.4710
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 30309440 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=7.8039398737250725;dx=-2.9508259195222918E-5
New Minimum: 7.8039398737250725 > 7.155321852680188
END: th(403200.00000000006)=7.155321852680188; dx=-2.731954847863814E-6 evalInputDelta=0.6486180210448849
Fitness changed from 7.8039398737250725 to 7.155321852680188
Iteration 3 complete. Error: 7.155321852680188 Total: 2.2732; Orientation: 0.0158; Line Search: 1.4945
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 5973b3cc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.849e+03, gradient 5.345e-03, dot -0.253; [07ece5cc-0b28-4432-b4a5-d1f249f803af = 1.000/1.000e+00]
Orientation rejected. Popping history element from 7.155321852680188, 7.8039398737250725, 8.894850295420973, 9.856489925517568
LBFGS Accumulation History: 3 points
Removed measurement 5973b3cc to history. Total: 3
th(0)=7.155321852680188;dx=-2.8513336360803573E-5
New Minimum: 7.155321852680188 > 6.789569025019858
WOLF (strong): th(868668.0670208557)=6.789569025019858; dx=5.4802138711135366E-8 evalInputDelta=0.36575282766032924
New Minimum: 6.789569025019858 > 4.918127503498823
END: th(434334.03351042786)=4.918127503498823; dx=-3.7917337685968196E-6 evalInputDelta=2.2371943491813644
Fitness changed from 7.155321852680188 to 4.918127503498823
Iteration 4 complete. Error: 4.918127503498823 Total: 3.9256; Orientation: 0.2959; Line Search: 2.8764
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 4c64fb9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.984e+03, gradient 5.179e-03, dot -0.225; [07ece5cc-0b28-4432-b4a5-d1f249f803af = 1.000/1.000e+00]
Orientation rejected. Popping history element from 4.918127503498823, 7.8039398737250725, 8.894850295420973, 9.856489925517568
LBFGS Accumulation History: 3 points
Removed measurement 4c64fb9 to history. Total: 3
th(0)=4.918127503498823;dx=-2.6779119007713415E-5
Armijo: th(935744.3088563365)=5.215519502547947; dx=4.091246538694962E-7 evalInputDelta=-0.2973919990491236
New Minimum: 4.918127503498823 > 2.6041518745634953
END: th(467872.15442816826)=2.6041518745634953; dx=-3.868341954507646E-6 evalInputDelta=2.313975628935328
Fitness changed from 4.918127503498823 to 2.6041518745634953
Iteration 5 complete. Error: 2.6041518745634953 Total: 3.3582; Orientation: 0.2934; Line Search: 2.3534
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 782cee0b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.260e+03, gradient 5.279e-03, dot -0.202; [07ece5cc-0b28-4432-b4a5-d1f249f803af = 1.000/1.000e+00]
Orientation rejected. Popping history element from 2.6041518745634953, 7.8039398737250725, 8.894850295420973, 9.856489925517568
LBFGS Accumulation History: 3 points
Removed measurement 782cee0b to history. Total: 3
th(0)=2.6041518745634953;dx=-2.778096047569296E-5
Armijo: th(1008000.0000000003)=4.565124574706974; dx=1.2470813630136907E-6 evalInputDelta=-1.9609727001434791
New Minimum: 2.6041518745634953 > 0.5968331994541032
END: th(504000.0000000002)=0.5968331994541032; dx=-3.6009657376545875E-6 evalInputDelta=2.007318675109392
Fitness changed from 2.6041518745634953 to 0.5968331994541032
Iteration 6 complete. Error: 0.5968331994541032 Total: 3.5710; Orientation: 0.2995; Line Search: 2.5143
Adding measurement 7f93a95b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.154e+03, gradient 5.503e-03, dot -0.183; [07ece5cc-0b28-4432-b4a5-d1f249f803af = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.5968331994541032, 7.8039398737250725, 8.894850295420973, 9.856489925517568
LBFGS Accumulation History: 3 points
Removed measurement 7f93a95b to history. Total: 3
th(0)=0.5968331994541032;dx=-3.0221002387683124E-5
Armijo: th(1085835.0837760698)=4.708798252696653; dx=2.2763932155690964E-6 evalInputDelta=-4.11196505324255
New Minimum: 0.5968331994541032 > -0.9825651247929219
END: th(542917.5418880349)=-0.9825651247929219; dx=-3.3318060289930837E-6 evalInputDelta=1.579398324247025
Fitness changed from 0.5968331994541032 to -0.9825651247929219
Iteration 7 complete. Error: -0.9825651247929219 Total: 3.2545; Orientation: 0.2830; Line Search: 2.2805
Adding measurement 321d23d3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.181e+03, gradient 6.001e-03, dot -0.159; [07ece5cc-0b28-4432-b4a5-d1f249f803af = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.9825651247929219, 7.8039398737250725, 8.894850295420973, 9.856489925517568
LBFGS Accumulation History: 3 points
Removed measurement 321d23d3 to history. Total: 3
th(0)=-0.9825651247929219;dx=-3.596173828684081E-5
Armijo: th(1169680.3860704207)=5.347891223424467; dx=3.5396129254829076E-6 evalInputDelta=-6.330456348217389
New Minimum: -0.9825651247929219 > -1.9739143805173072
END: th(584840.1930352104)=-1.9739143805173072; dx=-2.5084290110892577E-6 evalInputDelta=0.9913492557243853
Fitness changed from -0.9825651247929219 to -1.9739143805173072
Iteration 8 complete. Error: -1.9739143805173072 Total: 3.6744; Orientation: 0.2918; Line Search: 2.6440
Adding measurement f7dc39d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.473e+03, gradient 5.977e-03, dot -0.140; [07ece5cc-0b28-4432-b4a5-d1f249f803af = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.9739143805173072, 7.8039398737250725, 8.894850295420973, 9.856489925517568
LBFGS Accumulation History: 3 points
Removed measurement f7dc39d to history. Total: 3
th(0)=-1.9739143805173072;dx=-3.5675535728363276E-5
Armijo: th(1260000.0000000005)=4.777052357405706; dx=3.712499487123266E-6 evalInputDelta=-6.750966737923013
New Minimum: -1.9739143805173072 > -3.60253124650594
END: th(630000.0000000002)=-3.60253124650594; dx=-3.0611399796384507E-6 evalInputDelta=1.6286168659886329
Fitness changed from -1.9739143805173072 to -3.60253124650594
Iteration 9 complete. Error: -3.60253124650594 Total: 3.1793; Orientation: 0.2807; Line Search: 2.1681
Adding measurement 5f447abb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.148e+03, gradient 6.149e-03, dot -0.130; [07ece5cc-0b28-4432-b4a5-d1f249f803af = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.60253124650594, 7.8039398737250725, 8.894850295420973, 9.856489925517568
LBFGS Accumulation History: 3 points
Removed measurement 5f447abb to history. Total: 3
th(0)=-3.60253124650594;dx=-3.773944741495898E-5
Armijo: th(1357293.8547200873)=5.557219073455533; dx=4.533246922711523E-6 evalInputDelta=-9.159750319961473
New Minimum: -3.60253124650594 > -4.893399626387406
END: th(678646.9273600436)=-4.893399626387406; dx=-2.332252169513614E-6 evalInputDelta=1.2908683798814664
Fitness changed from -3.60253124650594 to -4.893399626387406
Iteration 10 complete. Error: -4.893399626387406 Total: 3.3341; Orientation: 0.2939; Line Search: 2.2753
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 40ecb1f1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.030e+04, gradient 6.440e-03, dot -0.112; [07ece5cc-0b28-4432-b4a5-d1f249f803af = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.893399626387406, 7.8039398737250725, 8.894850295420973, 9.856489925517568
LBFGS Accumulation History: 3 points
Removed measurement 40ecb1f1 to history. Total: 3
th(0)=-4.893399626387406;dx=-4.1432840136832837E-5
Armijo: th(1462100.482588026)=7.0657817646113195; dx=5.761164296431057E-6 evalInputDelta=-11.959181390998726
New Minimum: -4.893399626387406 > -5.923500303110227
END: th(731050.241294013)=-5.923500303110227; dx=-1.6582994119506172E-6 evalInputDelta=1.030100676722821
Fitness changed from -4.893399626387406 to -5.923500303110227
Iteration 11 complete. Error: -5.923500303110227 Total: 3.5318; Orientation: 0.2856; Line Search: 2.2228
Adding measurement 8bdf7a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.116e+04, gradient 6.562e-03, dot -0.101; [07ece5cc-0b28-4432-b4a5-d1f249f803af = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.923500303110227, 7.8039398737250725, 8.894850295420973, 9.856489925517568
LBFGS Accumulation History: 3 points
Removed measurement 8bdf7a to history. Total: 3
th(0)=-5.923500303110227;dx=-4.3008454444917467E-5
Armijo: th(1575000.0000000007)=7.94285949685624; dx=6.292387217514334E-6 evalInputDelta=-13.866359799966467
New Minimum: -5.923500303110227 > -6.73937894467306
END: th(787500.0000000003)=-6.73937894467306; dx=-1.4868530485100248E-6 evalInputDelta=0.8158786415628327
Fitness changed from -5.923500303110227 to -6.73937894467306
Iteration 12 complete. Error: -6.73937894467306 Total: 3.1710; Orientation: 0.2925; Line Search: 2.2152
Adding measurement 40bafa86 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.308e+04, gradient 6.619e-03, dot -0.090; [07ece5cc-0b28-4432-b4a5-d1f249f803af = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.73937894467306, 7.8039398737250725, 8.894850295420973, 9.856489925517568
LBFGS Accumulation History: 3 points
Removed measurement 40bafa86 to history. Total: 3
th(0)=-6.73937894467306;dx=-4.3732365239336124E-5
Armijo: th(1696617.3184001092)=9.242390506571862; dx=7.204011103647941E-6 evalInputDelta=-15.981769451244922
New Minimum: -6.73937894467306 > -7.231592651295042
END: th(848308.6592000546)=-7.231592651295042; dx=-1.312491396055782E-6 evalInputDelta=0.49221370662198183
Fitness changed from -6.73937894467306 to -7.231592651295042
Iteration 13 complete. Error: -7.231592651295042 Total: 3.4765; Orientation: 0.4440; Line Search: 2.3013
Adding measurement 74f5893 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.448e+04, gradient 6.701e-03, dot -0.080; [07ece5cc-0b28-4432-b4a5-d1f249f803af = 1.000/1.000e+00]
Orientation rejected. Popping history element from -7.231592651295042, 7.8039398737250725, 8.894850295420973, 9.856489925517568
LBFGS Accumulation History: 3 points
Removed measurement 74f5893 to history. Total: 3
th(0)=-7.231592651295042;dx=-4.484549304252495E-5
Armijo: th(1827625.6032350326)=10.64265693991506; dx=7.3196601369298234E-6 evalInputDelta=-17.874249591210102
New Minimum: -7.231592651295042 > -7.4281594880142805
END: th(913812.8016175163)=-7.4281594880142805; dx=-1.13687327956611E-6 evalInputDelta=0.19656683671923858
Fitness changed from -7.231592651295042 to -7.4281594880142805
Iteration 14 complete. Error: -7.4281594880142805 Total: 3.2630; Orientation: 0.3067; Line Search: 2.3116
Adding measurement 4d1bdb71 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.610e+04, gradient 6.715e-03, dot -0.072; [07ece5cc-0b28-4432-b4a5-d1f249f803af = 1.000/1.000e+00]
Orientation rejected. Popping history element from -7.4281594880142805, 7.8039398737250725, 8.894850295420973, 9.856489925517568
LBFGS Accumulation History: 3 points
Removed measurement 4d1bdb71 to history. Total: 3
th(0)=-7.4281594880142805;dx=-4.5006138940177596E-5
Armijo: th(1968750.0000000012)=12.065290575824683; dx=7.656280563221717E-6 evalInputDelta=-19.493450063838964
New Minimum: -7.4281594880142805 > -7.982467965648297
END: th(984375.0000000006)=-7.982467965648297; dx=-1.1723918013134282E-6 evalInputDelta=0.5543084776340166
Fitness changed from -7.4281594880142805 to -7.982467965648297
Iteration 15 complete. Error: -7.982467965648297 Total: 3.4651; Orientation: 0.2956; Line Search: 2.4414
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -7.982467965648297 (> -Infinity) after 55.168s (< 5400.000s)
Returns
-7.982467965648297