BasicOptimizer.scala:89 executed in 44.96 seconds (0.803 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 215028504892300
Reset training subject: 215029118675500
Adding measurement 509d3703 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-3.983543322960314;dx=-9.625014118329824E-5
New Minimum: -3.983543322960314 > -3.983705809689718
WOLFE (weak): th(2.154434690031884)=-3.983705809689718; dx=-9.668292013236453E-5 evalInputDelta=1.6248672940388786E-4
New Minimum: -3.983705809689718 > -3.9838687268355706
WOLFE (weak): th(4.308869380063768)=-3.9838687268355706; dx=-9.690646219181153E-5 evalInputDelta=3.254038752564803E-4
New Minimum: -3.9838687268355706 > -3.9845195945983996
WOLFE (weak): th(12.926608140191302)=-3.9845195945983996; dx=-9.65658278486672E-5 evalInputDelta=9.762716380854464E-4
New Minimum: -3.9845195945983996 > -3.9874540057953256
WOLFE (weak): th(51.70643256076521)=-3.9874540057953256; dx=-9.645457897074558E-5 evalInputDelta=0.003910682835011503
New Minimum: -3.9874540057953256 > -4.0031499417542005
WOLFE (weak): th(258.53216280382605)=-4.0031499417542005; dx=-9.665721779346126E-5 evalInputDelta=0.019606618793886366
New Minimum: -4.0031499417542005 > -4.101366765748564
WOLFE (weak): th(1551.1929768229563)=-4.101366765748564; dx=-9.704261794606586E-5 evalInputDelta=0.11782344278824963
New Minimum: -4.101366765748564 > -4.813349409423029
WOLFE (weak): th(10858.350837760694)=-4.813349409423029; dx=-9.517598574007517E-5 evalInputDelta=0.8298060864627153
New Minimum: -4.813349409423029 > -7.712351610626442
END: th(86866.80670208555)=-7.712351610626442; dx=-3.3862782235688796E-5 evalInputDelta=3.7288082876661277
Fitness changed from -3.983543322960314 to -7.712351610626442
Iteration 1 complete. Error: -7.712351610626442 Total: 7.3987; Orientation: 0.0156; Line Search: 5.6459
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 28bdad56 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-7.712351610626442;dx=-1.5709093777203706E-4
New Minimum: -7.712351610626442 > -9.878562878015138
END: th(187148.86177126726)=-9.878562878015138; dx=-1.1675162579889861E-5 evalInputDelta=2.1662112673886966
Fitness changed from -7.712351610626442 to -9.878562878015138
Iteration 2 complete. Error: -9.878562878015138 Total: 1.8295; Orientation: 0.0164; Line Search: 1.2182
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 2ed07190 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-9.878562878015138;dx=-1.211339877159888E-4
New Minimum: -9.878562878015138 > -10.366604662373994
END: th(403200.00000000006)=-10.366604662373994; dx=-3.9062374567939734E-6 evalInputDelta=0.48804178435885603
Fitness changed from -9.878562878015138 to -10.366604662373994
Iteration 3 complete. Error: -10.366604662373994 Total: 1.8826; Orientation: 0.0161; Line Search: 1.2489
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 669c6b4f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.264e+03, gradient 1.007e-02, dot -0.204; [c6de3d10-5e55-419f-aab1-a3d6169f2e19 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.366604662373994, -9.878562878015138, -7.712351610626442, -3.983543322960314
LBFGS Accumulation History: 3 points
Removed measurement 669c6b4f to history. Total: 3
th(0)=-10.366604662373994;dx=-1.012310436933833E-4
Armijo: th(868668.0670208557)=-2.149520640401864; dx=4.811909324948955E-6 evalInputDelta=-8.21708402197213
New Minimum: -10.366604662373994 > -14.397504520464931
END: th(434334.03351042786)=-14.397504520464931; dx=-1.1915449323510406E-5 evalInputDelta=4.030899858090937
Fitness changed from -10.366604662373994 to -14.397504520464931
Iteration 4 complete. Error: -14.397504520464931 Total: 2.7333; Orientation: 0.2950; Line Search: 1.8654
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 1a2fdb45 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.591e+03, gradient 1.002e-02, dot -0.192; [c6de3d10-5e55-419f-aab1-a3d6169f2e19 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -14.397504520464931, -9.878562878015138, -7.712351610626442, -3.983543322960314
LBFGS Accumulation History: 3 points
Removed measurement 1a2fdb45 to history. Total: 3
th(0)=-14.397504520464931;dx=-1.0023049587052919E-4
Armijo: th(935744.3088563365)=-0.6697132654726197; dx=5.886139051127269E-6 evalInputDelta=-13.727791254992312
New Minimum: -14.397504520464931 > -17.140141438126236
END: th(467872.15442816826)=-17.140141438126236; dx=-1.1787183236041013E-5 evalInputDelta=2.7426369176613044
Fitness changed from -14.397504520464931 to -17.140141438126236
Iteration 5 complete. Error: -17.140141438126236 Total: 2.9960; Orientation: 0.2862; Line Search: 1.8964
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 75075bee to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.532e+03, gradient 1.037e-02, dot -0.174; [c6de3d10-5e55-419f-aab1-a3d6169f2e19 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.140141438126236, -9.878562878015138, -7.712351610626442, -3.983543322960314
LBFGS Accumulation History: 3 points
Removed measurement 75075bee to history. Total: 3
th(0)=-17.140141438126236;dx=-1.0742889674033082E-4
Armijo: th(1008000.0000000003)=0.6960131810817884; dx=6.67638199010786E-6 evalInputDelta=-17.836154619208024
New Minimum: -17.140141438126236 > -19.123680027852558
END: th(504000.0000000002)=-19.123680027852558; dx=-1.106447037846803E-5 evalInputDelta=1.9835385897263222
Fitness changed from -17.140141438126236 to -19.123680027852558
Iteration 6 complete. Error: -19.123680027852558 Total: 2.7451; Orientation: 0.3080; Line Search: 1.8449
Adding measurement e5b370d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.422e+03, gradient 1.019e-02, dot -0.157; [c6de3d10-5e55-419f-aab1-a3d6169f2e19 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.123680027852558, -9.878562878015138, -7.712351610626442, -3.983543322960314
LBFGS Accumulation History: 3 points
Removed measurement e5b370d to history. Total: 3
th(0)=-19.123680027852558;dx=-1.0372022179044001E-4
Armijo: th(1085835.0837760698)=3.609639793865753; dx=6.215023859715177E-6 evalInputDelta=-22.73331982171831
New Minimum: -19.123680027852558 > -19.82480073545547
END: th(542917.5418880349)=-19.82480073545547; dx=-1.134437152357708E-5 evalInputDelta=0.7011207076029109
Fitness changed from -19.123680027852558 to -19.82480073545547
Iteration 7 complete. Error: -19.82480073545547 Total: 2.6408; Orientation: 0.2921; Line Search: 1.7717
Adding measurement 306a1d0c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.004e+03, gradient 1.020e-02, dot -0.141; [c6de3d10-5e55-419f-aab1-a3d6169f2e19 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.82480073545547, -9.878562878015138, -7.712351610626442, -3.983543322960314
LBFGS Accumulation History: 3 points
Removed measurement 306a1d0c to history. Total: 3
th(0)=-19.82480073545547;dx=-1.039759932274678E-4
Armijo: th(1169680.3860704207)=5.469824085498246; dx=7.616528373391253E-6 evalInputDelta=-25.294624820953715
New Minimum: -19.82480073545547 > -21.32273836047998
END: th(584840.1930352104)=-21.32273836047998; dx=-1.2230300119139177E-5 evalInputDelta=1.4979376250245124
Fitness changed from -19.82480073545547 to -21.32273836047998
Iteration 8 complete. Error: -21.32273836047998 Total: 2.6837; Orientation: 0.2952; Line Search: 1.8457
Adding measurement 5f017fdd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.828e+03, gradient 1.020e-02, dot -0.129; [c6de3d10-5e55-419f-aab1-a3d6169f2e19 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -21.32273836047998, -9.878562878015138, -7.712351610626442, -3.983543322960314
LBFGS Accumulation History: 3 points
Removed measurement 5f017fdd to history. Total: 3
th(0)=-21.32273836047998;dx=-1.0384324974369351E-4
Armijo: th(1260000.0000000005)=11.072140737191411; dx=7.285544686987436E-6 evalInputDelta=-32.39487909767139
Armijo: th(630000.0000000002)=-19.91668562498441; dx=-1.016122403084713E-5 evalInputDelta=-1.406052735495571
New Minimum: -21.32273836047998 > -28.29669974970382
END: th(210000.0000000001)=-28.29669974970382; dx=-5.1707519113633317E-5 evalInputDelta=6.973961389223838
Fitness changed from -21.32273836047998 to -28.29669974970382
Iteration 9 complete. Error: -28.29669974970382 Total: 3.5442; Orientation: 0.5069; Line Search: 2.5006
Adding measurement 3d38acd5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.151e+04, gradient 8.755e-03, dot -0.158; [c6de3d10-5e55-419f-aab1-a3d6169f2e19 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -28.29669974970382, -9.878562878015138, -7.712351610626442, -3.983543322960314
LBFGS Accumulation History: 3 points
Removed measurement 3d38acd5 to history. Total: 3
th(0)=-28.29669974970382;dx=-7.655222086801322E-5
New Minimum: -28.29669974970382 > -31.795363334824224
END: th(452431.2849066958)=-31.795363334824224; dx=-2.582548592035291E-5 evalInputDelta=3.498663585120404
Fitness changed from -28.29669974970382 to -31.795363334824224
Iteration 10 complete. Error: -31.795363334824224 Total: 2.0173; Orientation: 0.2933; Line Search: 1.1868
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 7cbb2a6a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.405e+04, gradient 9.439e-03, dot -0.148; [c6de3d10-5e55-419f-aab1-a3d6169f2e19 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -31.795363334824224, -9.878562878015138, -7.712351610626442, -3.983543322960314
LBFGS Accumulation History: 3 points
Removed measurement 7cbb2a6a to history. Total: 3
th(0)=-31.795363334824224;dx=-8.901315753736588E-5
Armijo: th(974733.6550586841)=-13.577767410721883; dx=1.0764130850539868E-7 evalInputDelta=-18.21759592410234
New Minimum: -31.795363334824224 > -32.27450887475659
END: th(487366.82752934203)=-32.27450887475659; dx=-1.6521373805611695E-5 evalInputDelta=0.4791455399323681
Fitness changed from -31.795363334824224 to -32.27450887475659
Iteration 11 complete. Error: -32.27450887475659 Total: 2.7640; Orientation: 0.2950; Line Search: 1.8786
Adding measurement 4518e5c5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.675e+04, gradient 9.671e-03, dot -0.140; [c6de3d10-5e55-419f-aab1-a3d6169f2e19 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -32.27450887475659, -9.878562878015138, -7.712351610626442, -3.983543322960314
LBFGS Accumulation History: 3 points
Removed measurement 4518e5c5 to history. Total: 3
th(0)=-32.27450887475659;dx=-9.343148570957719E-5
Armijo: th(1050000.0000000005)=-8.286530090745252; dx=5.423244593338475E-6 evalInputDelta=-23.98797878401134
Armijo: th(525000.0000000002)=-31.502346082143276; dx=-1.1804306366927583E-5 evalInputDelta=-0.7721627926133152
New Minimum: -32.27450887475659 > -37.044024805571425
END: th(175000.0000000001)=-37.044024805571425; dx=-4.63482651776662E-5 evalInputDelta=4.769515930814833
Fitness changed from -32.27450887475659 to -37.044024805571425
Iteration 12 complete. Error: -37.044024805571425 Total: 3.6159; Orientation: 0.2926; Line Search: 2.7343
Adding measurement 768eebfd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.459e+04, gradient 8.351e-03, dot -0.176; [c6de3d10-5e55-419f-aab1-a3d6169f2e19 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -37.044024805571425, -9.878562878015138, -7.712351610626442, -3.983543322960314
LBFGS Accumulation History: 3 points
Removed measurement 768eebfd to history. Total: 3
th(0)=-37.044024805571425;dx=-6.96763772105078E-5
New Minimum: -37.044024805571425 > -39.32740559746124
END: th(377026.07075557986)=-39.32740559746124; dx=-2.654136270607809E-5 evalInputDelta=2.2833807918898117
Fitness changed from -37.044024805571425 to -39.32740559746124
Iteration 13 complete. Error: -39.32740559746124 Total: 2.0991; Orientation: 0.2831; Line Search: 1.2440
Adding measurement 4ed5e78b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.202e+04, gradient 8.516e-03, dot -0.174; [c6de3d10-5e55-419f-aab1-a3d6169f2e19 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -39.32740559746124, -9.878562878015138, -7.712351610626442, -3.983543322960314
LBFGS Accumulation History: 3 points
Removed measurement 4ed5e78b to history. Total: 3
th(0)=-39.32740559746124;dx=-7.241734897005872E-5
Armijo: th(812278.0458822368)=-29.478029833563568; dx=-4.677641786526453E-6 evalInputDelta=-9.849375763897669
New Minimum: -39.32740559746124 > -40.632047105711806
END: th(406139.0229411184)=-40.632047105711806; dx=-1.940201924434585E-5 evalInputDelta=1.304641508250569
Fitness changed from -39.32740559746124 to -40.632047105711806
Iteration 14 complete. Error: -40.632047105711806 Total: 2.6773; Orientation: 0.2954; Line Search: 1.8376
Adding measurement 424d704e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.593e+04, gradient 8.930e-03, dot -0.170; [c6de3d10-5e55-419f-aab1-a3d6169f2e19 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -40.632047105711806, -9.878562878015138, -7.712351610626442, -3.983543322960314
LBFGS Accumulation History: 3 points
Removed measurement 424d704e to history. Total: 3
th(0)=-40.632047105711806;dx=-7.965863913270472E-5
Armijo: th(875000.0000000006)=-24.82073417928953; dx=2.3847594284576326E-6 evalInputDelta=-15.811312926422275
Armijo: th(437500.0000000003)=-40.32012571967334; dx=-1.1322378861732969E-5 evalInputDelta=-0.3119213860384633
New Minimum: -40.632047105711806 > -43.71947250833695
END: th(145833.33333333343)=-43.71947250833695; dx=-4.0416963009140016E-5 evalInputDelta=3.0874254026251435
Fitness changed from -40.632047105711806 to -43.71947250833695
Iteration 15 complete. Error: -43.71947250833695 Total: 3.3144; Orientation: 0.2919; Line Search: 2.4595
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -43.71947250833695 (> -Infinity) after 44.963s (< 5400.000s)
Returns
-43.71947250833695