BasicOptimizer.scala:89 executed in 55.58 seconds (1.429 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 208562505882200
Reset training subject: 208563303011300
Adding measurement 608ce319 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=10.920527215477685;dx=-4.714328571827415E-5
New Minimum: 10.920527215477685 > 10.920445935162846
WOLFE (weak): th(2.154434690031884)=10.920445935162846; dx=-4.726843671431981E-5 evalInputDelta=8.128031483956022E-5
New Minimum: 10.920445935162846 > 10.920364478788915
WOLFE (weak): th(4.308869380063768)=10.920364478788915; dx=-4.726235477377726E-5 evalInputDelta=1.6273668876998215E-4
New Minimum: 10.920364478788915 > 10.920041917317832
WOLFE (weak): th(12.926608140191302)=10.920041917317832; dx=-4.736606702423763E-5 evalInputDelta=4.852981598535422E-4
New Minimum: 10.920041917317832 > 10.918588201851003
WOLFE (weak): th(51.70643256076521)=10.918588201851003; dx=-4.702534623995425E-5 evalInputDelta=0.00193901362668214
New Minimum: 10.918588201851003 > 10.910791155673706
WOLFE (weak): th(258.53216280382605)=10.910791155673706; dx=-4.761578755039216E-5 evalInputDelta=0.009736059803978847
New Minimum: 10.910791155673706 > 10.86116732550764
WOLFE (weak): th(1551.1929768229563)=10.86116732550764; dx=-4.8365702064527175E-5 evalInputDelta=0.05935988997004493
New Minimum: 10.86116732550764 > 10.480117117477352
WOLFE (weak): th(10858.350837760694)=10.480117117477352; dx=-5.0522993418851556E-5 evalInputDelta=0.44041009800033315
New Minimum: 10.480117117477352 > 8.010522332382196
END: th(86866.80670208555)=8.010522332382196; dx=-2.6264283041791503E-5 evalInputDelta=2.910004883095489
Fitness changed from 10.920527215477685 to 8.010522332382196
Iteration 1 complete. Error: 8.010522332382196 Total: 9.2925; Orientation: 0.0156; Line Search: 7.0477
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 64342b84 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=8.010522332382196;dx=-7.886953044203634E-5
New Minimum: 8.010522332382196 > 3.276455627080747
END: th(187148.86177126726)=3.276455627080747; dx=-1.850441000982237E-5 evalInputDelta=4.73406670530145
Fitness changed from 8.010522332382196 to 3.276455627080747
Iteration 2 complete. Error: 3.276455627080747 Total: 2.3146; Orientation: 0.0159; Line Search: 1.5366
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 5f5b45a0 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=3.276455627080747;dx=-1.1072628693997087E-4
New Minimum: 3.276455627080747 > -0.12472704803629675
END: th(403200.00000000006)=-0.12472704803629675; dx=-2.9918671779922156E-6 evalInputDelta=3.4011826751170435
Fitness changed from 3.276455627080747 to -0.12472704803629675
Iteration 3 complete. Error: -0.12472704803629675 Total: 2.7773; Orientation: 0.0160; Line Search: 1.9804
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 5915bbb6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.276e+03, gradient 1.138e-02, dot -0.125; [93601940-ff21-4164-a1fd-ebfe22c662f1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.12472704803629675, 3.276455627080747, 8.010522332382196, 10.920527215477685
LBFGS Accumulation History: 3 points
Removed measurement 5915bbb6 to history. Total: 3
th(0)=-0.12472704803629675;dx=-1.2916489217938123E-4
Armijo: th(868668.0670208557)=2.4928479829079997; dx=5.928174692172164E-6 evalInputDelta=-2.6175750309442964
New Minimum: -0.12472704803629675 > -6.662381744994878
END: th(434334.03351042786)=-6.662381744994878; dx=-4.926595726104216E-6 evalInputDelta=6.537654696958581
Fitness changed from -0.12472704803629675 to -6.662381744994878
Iteration 4 complete. Error: -6.662381744994878 Total: 3.3513; Orientation: 0.2980; Line Search: 2.3541
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement f7a9448 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.215e+03, gradient 1.047e-02, dot -0.111; [93601940-ff21-4164-a1fd-ebfe22c662f1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.662381744994878, 3.276455627080747, 8.010522332382196, 10.920527215477685
LBFGS Accumulation History: 3 points
Removed measurement f7a9448 to history. Total: 3
th(0)=-6.662381744994878;dx=-1.0937288979270486E-4
Armijo: th(935744.3088563365)=-1.6767427751946684; dx=6.643089842712323E-6 evalInputDelta=-4.98563896980021
New Minimum: -6.662381744994878 > -13.97570568034018
END: th(467872.15442816826)=-13.97570568034018; dx=-8.235163257275806E-6 evalInputDelta=7.313323935345302
Fitness changed from -6.662381744994878 to -13.97570568034018
Iteration 5 complete. Error: -13.97570568034018 Total: 3.5513; Orientation: 0.3225; Line Search: 2.4290
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 38147961 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.187e+03, gradient 1.056e-02, dot -0.108; [93601940-ff21-4164-a1fd-ebfe22c662f1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -13.97570568034018, 3.276455627080747, 8.010522332382196, 10.920527215477685
LBFGS Accumulation History: 3 points
Removed measurement 38147961 to history. Total: 3
th(0)=-13.97570568034018;dx=-1.1118081477612914E-4
Armijo: th(1008000.0000000003)=-3.6420992329251973; dx=8.417554534367438E-6 evalInputDelta=-10.333606447414983
New Minimum: -13.97570568034018 > -20.281344636245024
END: th(504000.0000000002)=-20.281344636245024; dx=-9.257244324604949E-6 evalInputDelta=6.305638955904843
Fitness changed from -13.97570568034018 to -20.281344636245024
Iteration 6 complete. Error: -20.281344636245024 Total: 3.3562; Orientation: 0.2903; Line Search: 2.2628
Adding measurement 17decef6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.033e+03, gradient 1.091e-02, dot -0.103; [93601940-ff21-4164-a1fd-ebfe22c662f1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -20.281344636245024, 3.276455627080747, 8.010522332382196, 10.920527215477685
LBFGS Accumulation History: 3 points
Removed measurement 17decef6 to history. Total: 3
th(0)=-20.281344636245024;dx=-1.1893882795498706E-4
Armijo: th(1085835.0837760698)=-4.552530288141828; dx=1.034673076625574E-5 evalInputDelta=-15.728814348103196
New Minimum: -20.281344636245024 > -24.838757761288186
END: th(542917.5418880349)=-24.838757761288186; dx=-6.117874370938905E-6 evalInputDelta=4.557413125043162
Fitness changed from -20.281344636245024 to -24.838757761288186
Iteration 7 complete. Error: -24.838757761288186 Total: 3.1968; Orientation: 0.3132; Line Search: 2.2040
Adding measurement 36bd4329 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.728e+03, gradient 1.117e-02, dot -0.099; [93601940-ff21-4164-a1fd-ebfe22c662f1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -24.838757761288186, 3.276455627080747, 8.010522332382196, 10.920527215477685
LBFGS Accumulation History: 3 points
Removed measurement 36bd4329 to history. Total: 3
th(0)=-24.838757761288186;dx=-1.2465631801688941E-4
Armijo: th(1169680.3860704207)=-4.551287444225437; dx=1.334039428372248E-5 evalInputDelta=-20.28747031706275
New Minimum: -24.838757761288186 > -28.90426617562511
END: th(584840.1930352104)=-28.90426617562511; dx=-4.0196818476649485E-6 evalInputDelta=4.065508414336925
Fitness changed from -24.838757761288186 to -28.90426617562511
Iteration 8 complete. Error: -28.90426617562511 Total: 3.5392; Orientation: 0.4854; Line Search: 2.3700
Adding measurement 21efc5f0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.215e+03, gradient 1.101e-02, dot -0.091; [93601940-ff21-4164-a1fd-ebfe22c662f1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -28.90426617562511, 3.276455627080747, 8.010522332382196, 10.920527215477685
LBFGS Accumulation History: 3 points
Removed measurement 21efc5f0 to history. Total: 3
th(0)=-28.90426617562511;dx=-1.2116529773547322E-4
Armijo: th(1260000.0000000005)=-7.78766798232364; dx=1.452601761531899E-5 evalInputDelta=-21.11659819330147
New Minimum: -28.90426617562511 > -33.59563257867007
END: th(630000.0000000002)=-33.59563257867007; dx=-3.745741096124414E-6 evalInputDelta=4.6913664030449596
Fitness changed from -28.90426617562511 to -33.59563257867007
Iteration 9 complete. Error: -33.59563257867007 Total: 3.1696; Orientation: 0.2853; Line Search: 2.1951
Adding measurement 2923e241 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.241e+04, gradient 1.082e-02, dot -0.090; [93601940-ff21-4164-a1fd-ebfe22c662f1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -33.59563257867007, 3.276455627080747, 8.010522332382196, 10.920527215477685
LBFGS Accumulation History: 3 points
Removed measurement 2923e241 to history. Total: 3
th(0)=-33.59563257867007;dx=-1.1705307253060463E-4
Armijo: th(1357293.8547200873)=-9.917392651940098; dx=1.4598975520817571E-5 evalInputDelta=-23.678239926729972
New Minimum: -33.59563257867007 > -38.609055625100495
END: th(678646.9273600436)=-38.609055625100495; dx=-1.8765157940034907E-6 evalInputDelta=5.013423046430425
Fitness changed from -33.59563257867007 to -38.609055625100495
Iteration 10 complete. Error: -38.609055625100495 Total: 3.5064; Orientation: 0.3166; Line Search: 2.4968
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 1b0b2af4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.246e+04, gradient 1.052e-02, dot -0.084; [93601940-ff21-4164-a1fd-ebfe22c662f1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -38.609055625100495, 3.276455627080747, 8.010522332382196, 10.920527215477685
LBFGS Accumulation History: 3 points
Removed measurement 1b0b2af4 to history. Total: 3
th(0)=-38.609055625100495;dx=-1.1055091704985807E-4
Armijo: th(1462100.482588026)=-10.135490775241834; dx=1.559890041906097E-5 evalInputDelta=-28.47356484985866
New Minimum: -38.609055625100495 > -42.32793801828194
END: th(731050.241294013)=-42.32793801828194; dx=-1.8602954638488811E-6 evalInputDelta=3.7188823931814454
Fitness changed from -38.609055625100495 to -42.32793801828194
Iteration 11 complete. Error: -42.32793801828194 Total: 3.1907; Orientation: 0.2828; Line Search: 2.2097
Adding measurement 6fb57025 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.653e+04, gradient 1.079e-02, dot -0.080; [93601940-ff21-4164-a1fd-ebfe22c662f1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -42.32793801828194, 3.276455627080747, 8.010522332382196, 10.920527215477685
LBFGS Accumulation History: 3 points
Removed measurement 6fb57025 to history. Total: 3
th(0)=-42.32793801828194;dx=-1.1634867602861055E-4
Armijo: th(1575000.0000000007)=-7.703594600094561; dx=1.702776829430146E-5 evalInputDelta=-34.62434341818738
New Minimum: -42.32793801828194 > -43.277380085837535
WOLF (strong): th(787500.0000000003)=-43.277380085837535; dx=1.4105910352288853E-6 evalInputDelta=0.9494420675555943
New Minimum: -43.277380085837535 > -53.34583207041457
END: th(262500.0000000001)=-53.34583207041457; dx=-4.6592348104320546E-5 evalInputDelta=11.017894052132633
Fitness changed from -42.32793801828194 to -53.34583207041457
Iteration 12 complete. Error: -53.34583207041457 Total: 4.2531; Orientation: 0.2947; Line Search: 3.2726
Adding measurement 6b66835a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.357e+04, gradient 9.111e-03, dot -0.111; [93601940-ff21-4164-a1fd-ebfe22c662f1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -53.34583207041457, 3.276455627080747, 8.010522332382196, 10.920527215477685
LBFGS Accumulation History: 3 points
Removed measurement 6b66835a to history. Total: 3
th(0)=-53.34583207041457;dx=-8.287550300636045E-5
New Minimum: -53.34583207041457 > -58.85395177648668
END: th(565539.1061333697)=-58.85395177648668; dx=-1.8771654024538666E-5 evalInputDelta=5.508119706072108
Fitness changed from -53.34583207041457 to -58.85395177648668
Iteration 13 complete. Error: -58.85395177648668 Total: 2.4691; Orientation: 0.2911; Line Search: 1.4530
Adding measurement 3a5e2abd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.111e+04, gradient 9.831e-03, dot -0.107; [93601940-ff21-4164-a1fd-ebfe22c662f1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -58.85395177648668, 3.276455627080747, 8.010522332382196, 10.920527215477685
LBFGS Accumulation History: 3 points
Removed measurement 3a5e2abd to history. Total: 3
th(0)=-58.85395177648668;dx=-9.652671887296858E-5
Armijo: th(1218417.068823355)=-37.440732864423595; dx=9.263692217441105E-6 evalInputDelta=-21.413218912063087
New Minimum: -58.85395177648668 > -59.893612460477584
END: th(609208.5344116775)=-59.893612460477584; dx=-7.948997390509534E-6 evalInputDelta=1.0396606839909026
Fitness changed from -58.85395177648668 to -59.893612460477584
Iteration 14 complete. Error: -59.893612460477584 Total: 3.5411; Orientation: 0.2941; Line Search: 2.5196
Adding measurement 43e0dba7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.558e+04, gradient 1.046e-02, dot -0.097; [93601940-ff21-4164-a1fd-ebfe22c662f1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -59.893612460477584, 3.276455627080747, 8.010522332382196, 10.920527215477685
LBFGS Accumulation History: 3 points
Removed measurement 43e0dba7 to history. Total: 3
th(0)=-59.893612460477584;dx=-1.0935496465387787E-4
Armijo: th(1312500.0000000007)=-30.746003322001613; dx=1.6358072521716712E-5 evalInputDelta=-29.14760913847597
New Minimum: -59.893612460477584 > -59.916273059096156
WOLF (strong): th(656250.0000000003)=-59.916273059096156; dx=7.324328374672788E-7 evalInputDelta=0.022660598618571726
New Minimum: -59.916273059096156 > -67.63444690339091
END: th(218750.00000000012)=-67.63444690339091; dx=-4.4892603567934635E-5 evalInputDelta=7.740834442913325
Fitness changed from -59.893612460477584 to -67.63444690339091
Iteration 15 complete. Error: -67.63444690339091 Total: 4.0470; Orientation: 0.2853; Line Search: 3.0450
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -67.63444690339091 (> -Infinity) after 55.580s (< 5400.000s)
Returns
-67.63444690339091