BasicOptimizer.scala:89 executed in 49.24 seconds (0.929 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 223955296205400
Reset training subject: 223955928356400
Adding measurement 58013005 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-180.38604436042164;dx=-0.05974158376973561
New Minimum: -180.38604436042164 > -180.48992013540283
WOLFE (weak): th(2.154434690031884)=-180.48992013540283; dx=-0.05966073709629133 evalInputDelta=0.10387577498119072
New Minimum: -180.48992013540283 > -180.59379532508234
WOLFE (weak): th(4.308869380063768)=-180.59379532508234; dx=-0.059743423076445465 evalInputDelta=0.20775096466070408
New Minimum: -180.59379532508234 > -181.00938734922238
WOLFE (weak): th(12.926608140191302)=-181.00938734922238; dx=-0.059411629623754285 evalInputDelta=0.6233429888007436
New Minimum: -181.00938734922238 > -182.87719340233818
WOLFE (weak): th(51.70643256076521)=-182.87719340233818; dx=-0.059418551685827534 evalInputDelta=2.491149041916543
New Minimum: -182.87719340233818 > -192.75215160989546
WOLFE (weak): th(258.53216280382605)=-192.75215160989546; dx=-0.05807846969235838 evalInputDelta=12.366107249473828
New Minimum: -192.75215160989546 > -246.35603402115825
END: th(1551.1929768229563)=-246.35603402115825; dx=-0.0451641103232551 evalInputDelta=65.96998966073662
Fitness changed from -180.38604436042164 to -246.35603402115825
Iteration 1 complete. Error: -246.35603402115825 Total: 6.2888; Orientation: 0.0180; Line Search: 4.4759
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 27873f12 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-246.35603402115825;dx=-0.06694576875262748
New Minimum: -246.35603402115825 > -349.62323531077436
END: th(3341.943960201201)=-349.62323531077436; dx=-0.03016349914828058 evalInputDelta=103.26720128961611
Fitness changed from -246.35603402115825 to -349.62323531077436
Iteration 2 complete. Error: -349.62323531077436 Total: 1.9040; Orientation: 0.0171; Line Search: 1.2771
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 6b01ca1a to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-349.62323531077436;dx=-0.07609341825015864
New Minimum: -349.62323531077436 > -399.49776701328005
END: th(7200.000000000001)=-399.49776701328005; dx=-0.013685647391044893 evalInputDelta=49.87453170250569
Fitness changed from -349.62323531077436 to -399.49776701328005
Iteration 3 complete. Error: -399.49776701328005 Total: 2.1720; Orientation: 0.0163; Line Search: 1.5279
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 3e57d9aa to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.938e+03, gradient 2.874e-01, dot -0.221; [1d8482f5-24ae-4c5b-b753-b8894e036034 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -399.49776701328005, -349.62323531077436, -246.35603402115825, -180.38604436042164
LBFGS Accumulation History: 3 points
Removed measurement 3e57d9aa to history. Total: 3
th(0)=-399.49776701328005;dx=-0.08228648859417477
Armijo: th(15511.929768229566)=-384.4408516257357; dx=-0.005863342290311346 evalInputDelta=-15.056915387544336
New Minimum: -399.49776701328005 > -478.9918508656517
END: th(7755.964884114783)=-478.9918508656517; dx=-0.016350906565229868 evalInputDelta=79.49408385237166
Fitness changed from -399.49776701328005 to -478.9918508656517
Iteration 4 complete. Error: -478.9918508656517 Total: 3.0345; Orientation: 0.3268; Line Search: 1.9920
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 2da92d4b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.025e+04, gradient 2.916e-01, dot -0.194; [1d8482f5-24ae-4c5b-b753-b8894e036034 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -478.9918508656517, -349.62323531077436, -246.35603402115825, -180.38604436042164
LBFGS Accumulation History: 3 points
Removed measurement 2da92d4b to history. Total: 3
th(0)=-478.9918508656517;dx=-0.08483246692293886
Armijo: th(16709.71980100601)=-469.7096022011326; dx=-0.005060436643275384 evalInputDelta=-9.282248664519102
New Minimum: -478.9918508656517 > -604.04724235884
END: th(8354.859900503005)=-604.04724235884; dx=-0.018148714619664927 evalInputDelta=125.05539149318827
Fitness changed from -478.9918508656517 to -604.04724235884
Iteration 5 complete. Error: -604.04724235884 Total: 2.9093; Orientation: 0.3221; Line Search: 1.9058
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 245c45dd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.429e+04, gradient 2.912e-01, dot -0.210; [1d8482f5-24ae-4c5b-b753-b8894e036034 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -604.04724235884, -349.62323531077436, -246.35603402115825, -180.38604436042164
LBFGS Accumulation History: 3 points
Removed measurement 245c45dd to history. Total: 3
th(0)=-604.04724235884;dx=-0.08457063682432674
Armijo: th(18000.000000000007)=-552.7014545546264; dx=-0.0032828123308856474 evalInputDelta=-51.345787804213614
New Minimum: -604.04724235884 > -739.5266856297764
END: th(9000.000000000004)=-739.5266856297764; dx=-0.019202187976880288 evalInputDelta=135.47944327093637
Fitness changed from -604.04724235884 to -739.5266856297764
Iteration 6 complete. Error: -739.5266856297764 Total: 3.0119; Orientation: 0.3643; Line Search: 1.9984
Adding measurement 2fb7b090 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.737e+04, gradient 2.926e-01, dot -0.197; [1d8482f5-24ae-4c5b-b753-b8894e036034 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -739.5266856297764, -349.62323531077436, -246.35603402115825, -180.38604436042164
LBFGS Accumulation History: 3 points
Removed measurement 2fb7b090 to history. Total: 3
th(0)=-739.5266856297764;dx=-0.08536597169485774
Armijo: th(19389.912210286962)=-635.5097447718307; dx=-0.0023145104465852437 evalInputDelta=-104.01694085794566
New Minimum: -739.5266856297764 > -869.7436343435595
END: th(9694.956105143481)=-869.7436343435595; dx=-0.018797395398163108 evalInputDelta=130.21694871378315
Fitness changed from -739.5266856297764 to -869.7436343435595
Iteration 7 complete. Error: -869.7436343435595 Total: 3.4069; Orientation: 0.3288; Line Search: 2.4451
Adding measurement 7185dce2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.553e+04, gradient 2.963e-01, dot -0.197; [1d8482f5-24ae-4c5b-b753-b8894e036034 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -869.7436343435595, -349.62323531077436, -246.35603402115825, -180.38604436042164
LBFGS Accumulation History: 3 points
Removed measurement 7185dce2 to history. Total: 3
th(0)=-869.7436343435595;dx=-0.0875439623570194
Armijo: th(20887.149751257515)=-713.4121374127593; dx=-7.835740991872383E-4 evalInputDelta=-156.3314969308002
New Minimum: -869.7436343435595 > -998.0001001990145
END: th(10443.574875628758)=-998.0001001990145; dx=-0.015581596171626631 evalInputDelta=128.25646585545496
Fitness changed from -869.7436343435595 to -998.0001001990145
Iteration 8 complete. Error: -998.0001001990145 Total: 3.0073; Orientation: 0.3151; Line Search: 2.0582
Adding measurement 133a76e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.347e+04, gradient 3.020e-01, dot -0.183; [1d8482f5-24ae-4c5b-b753-b8894e036034 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -998.0001001990145, -349.62323531077436, -246.35603402115825, -180.38604436042164
LBFGS Accumulation History: 3 points
Removed measurement 133a76e to history. Total: 3
th(0)=-998.0001001990145;dx=-0.0906595305335357
Armijo: th(22500.00000000001)=-799.526693383207; dx=-3.229913463917351E-4 evalInputDelta=-198.4734068158075
New Minimum: -998.0001001990145 > -1124.7531888014098
END: th(11250.000000000005)=-1124.7531888014098; dx=-0.013895902330824748 evalInputDelta=126.75308860239534
Fitness changed from -998.0001001990145 to -1124.7531888014098
Iteration 9 complete. Error: -1124.7531888014098 Total: 3.0269; Orientation: 0.3512; Line Search: 2.0100
Adding measurement 3dcf1f85 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.370e+04, gradient 3.060e-01, dot -0.189; [1d8482f5-24ae-4c5b-b753-b8894e036034 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1124.7531888014098, -349.62323531077436, -246.35603402115825, -180.38604436042164
LBFGS Accumulation History: 3 points
Removed measurement 3dcf1f85 to history. Total: 3
th(0)=-1124.7531888014098;dx=-0.09320860410135616
Armijo: th(24237.390262858706)=-875.9033615556791; dx=0.0011339228668663629 evalInputDelta=-248.8498272457307
New Minimum: -1124.7531888014098 > -1249.767243348607
END: th(12118.695131429353)=-1249.767243348607; dx=-0.012898546242340766 evalInputDelta=125.01405454719725
Fitness changed from -1124.7531888014098 to -1249.767243348607
Iteration 10 complete. Error: -1249.767243348607 Total: 2.6134; Orientation: 0.2996; Line Search: 1.7768
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 40fa7816 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.839e+05, gradient 3.076e-01, dot -0.190; [1d8482f5-24ae-4c5b-b753-b8894e036034 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1249.767243348607, -349.62323531077436, -246.35603402115825, -180.38604436042164
LBFGS Accumulation History: 3 points
Removed measurement 40fa7816 to history. Total: 3
th(0)=-1249.767243348607;dx=-0.09431253781773288
Armijo: th(26108.937189071898)=-991.6597361357478; dx=0.002300371408679328 evalInputDelta=-258.1075072128592
New Minimum: -1249.767243348607 > -1393.3467471883084
END: th(13054.468594535949)=-1393.3467471883084; dx=-0.014158853925645504 evalInputDelta=143.57950383970137
Fitness changed from -1249.767243348607 to -1393.3467471883084
Iteration 11 complete. Error: -1393.3467471883084 Total: 2.9630; Orientation: 0.2986; Line Search: 2.1109
Adding measurement 56ace25f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.546e+06, gradient 3.075e-01, dot -0.194; [1d8482f5-24ae-4c5b-b753-b8894e036034 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1393.3467471883084, -349.62323531077436, -246.35603402115825, -180.38604436042164
LBFGS Accumulation History: 3 points
Removed measurement 56ace25f to history. Total: 3
th(0)=-1393.3467471883084;dx=-0.09431957068187506
Armijo: th(28125.00000000002)=-1059.6713937665866; dx=0.0020101208903717527 evalInputDelta=-333.67535342172187
New Minimum: -1393.3467471883084 > -1514.5425183644381
END: th(14062.50000000001)=-1514.5425183644381; dx=-0.014251944795746405 evalInputDelta=121.19577117612971
Fitness changed from -1393.3467471883084 to -1514.5425183644381
Iteration 12 complete. Error: -1514.5425183644381 Total: 2.6464; Orientation: 0.2844; Line Search: 1.8007
Adding measurement 6f8246fc to history. Total: 3
Accepted: LBFGS Orientation magnitude: 2.325e+04, gradient 3.088e-01, dot 0.283; [1d8482f5-24ae-4c5b-b753-b8894e036034 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-1514.5425183644381;dx=-2033.648626229143
Armijo: th(30296.737828573387)=416.89489875156124; dx=0.02619778770750567 evalInputDelta=-1931.4374171159993
Armijo: th(15148.368914286693)=416.67964286873666; dx=0.05192972110002575 evalInputDelta=-1931.2221612331748
Armijo: th(5049.456304762231)=415.90877999370184; dx=0.15804420937723856 evalInputDelta=-1930.4512983581399
Armijo: th(1262.3640761905579)=411.4206970964717; dx=0.6255195514277295 evalInputDelta=-1925.9632154609099
Armijo: th(252.47281523811156)=392.77058496826396; dx=3.0393134073839656 evalInputDelta=-1907.3131033327022
Armijo: th(42.07880253968526)=298.9011965883178; dx=16.44847182796795 evalInputDelta=-1813.443714952756
Armijo: th(6.011257505669323)=-124.52553755428107; dx=64.3592152677886 evalInputDelta=-1390.016980810157
Armijo: th(0.7514071882086654)=-1315.7843537645554; dx=-228.8054582918362 evalInputDelta=-198.75816459988278
New Minimum: -1514.5425183644381 > -1567.285116106083
END: th(0.08348968757874059)=-1567.285116106083; dx=-1197.5378460225702 evalInputDelta=52.742597741644886
Fitness changed from -1514.5425183644381 to -1567.285116106083
Iteration 13 complete. Error: -1567.285116106083 Total: 7.1113; Orientation: 0.2949; Line Search: 6.2778
Adding measurement 1fc648b5 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 7.779e+03, gradient 3.055e-01, dot -0.032; [1d8482f5-24ae-4c5b-b753-b8894e036034 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1567.285116106083, -1514.5425183644381, -349.62323531077436, -246.35603402115825, -180.38604436042164
Rejected: LBFGS Orientation magnitude: 1.293e+05, gradient 3.055e-01, dot -0.192; [1d8482f5-24ae-4c5b-b753-b8894e036034 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1567.285116106083, -1514.5425183644381, -349.62323531077436, -246.35603402115825
LBFGS Accumulation History: 3 points
Removed measurement 1fc648b5 to history. Total: 4
Removed measurement 6f8246fc to history. Total: 3
th(0)=-1567.285116106083;dx=-0.09317742627310112
New Minimum: -1567.285116106083 > -1567.2939506017726
END: th(0.1798730791795628)=-1567.2939506017726; dx=-0.08130312973170584 evalInputDelta=0.008834495689598043
Fitness changed from -1567.285116106083 to -1567.2939506017726
Iteration 14 complete. Error: -1567.2939506017726 Total: 2.8954; Orientation: 0.7259; Line Search: 1.2806
Adding measurement 19c0ca6f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.631e+07, gradient 3.056e-01, dot -0.172; [1d8482f5-24ae-4c5b-b753-b8894e036034 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1567.2939506017726, -349.62323531077436, -246.35603402115825, -180.38604436042164
LBFGS Accumulation History: 3 points
Removed measurement 19c0ca6f to history. Total: 3
th(0)=-1567.2939506017726;dx=-0.09315849856880329
New Minimum: -1567.2939506017726 > -1567.3130862778482
END: th(0.3875248015873019)=-1567.3130862778482; dx=-0.08124277079523337 evalInputDelta=0.019135676075620722
Fitness changed from -1567.2939506017726 to -1567.3130862778482
Iteration 15 complete. Error: -1567.3130862778482 Total: 2.2250; Orientation: 0.3108; Line Search: 1.3386
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -1567.3130862778482 (> -Infinity) after 49.241s (< 5400.000s)
Returns
-1567.3130862778482