BasicOptimizer.scala:89 executed in 62.25 seconds (1.878 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 210627066830000
Reset training subject: 210627890256000
Adding measurement 172b937c to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=8.134354798372286;dx=-0.027278964402226
New Minimum: 8.134354798372286 > 8.095373006610126
WOLFE (weak): th(2.154434690031884)=8.095373006610126; dx=-0.027588955721269015 evalInputDelta=0.038981791762159546
New Minimum: 8.095373006610126 > 8.055953949135702
WOLFE (weak): th(4.308869380063768)=8.055953949135702; dx=-0.027574378752723352 evalInputDelta=0.07840084923658353
New Minimum: 8.055953949135702 > 7.893138959519957
WOLFE (weak): th(12.926608140191302)=7.893138959519957; dx=-0.02918705297384132 evalInputDelta=0.24121583885232845
New Minimum: 7.893138959519957 > 7.095223404069206
WOLFE (weak): th(51.70643256076521)=7.095223404069206; dx=-0.03209121861540895 evalInputDelta=1.0391313943030793
New Minimum: 7.095223404069206 > 2.2891703704044044
WOLFE (weak): th(258.53216280382605)=2.2891703704044044; dx=-0.03827267273931878 evalInputDelta=5.845184427967881
New Minimum: 2.2891703704044044 > -3.736914585272876
END: th(1551.1929768229563)=-3.736914585272876; dx=-0.012185014169374123 evalInputDelta=11.871269383645162
Fitness changed from 8.134354798372286 to -3.736914585272876
Iteration 1 complete. Error: -3.736914585272876 Total: 7.7563; Orientation: 0.0161; Line Search: 5.4885
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 319c7896 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-3.736914585272876;dx=-0.03983611306198609
New Minimum: -3.736914585272876 > -25.740148195871996
END: th(3341.943960201201)=-25.740148195871996; dx=-0.01096942859888597 evalInputDelta=22.003233610599118
Fitness changed from -3.736914585272876 to -25.740148195871996
Iteration 2 complete. Error: -25.740148195871996 Total: 2.3219; Orientation: 0.0160; Line Search: 1.5443
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 2b536aa to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-25.740148195871996;dx=-0.05289139894190538
New Minimum: -25.740148195871996 > -60.51469948895699
END: th(7200.000000000001)=-60.51469948895699; dx=-0.0040149474755412145 evalInputDelta=34.774551293085
Fitness changed from -25.740148195871996 to -60.51469948895699
Iteration 3 complete. Error: -60.51469948895699 Total: 2.2493; Orientation: 0.0159; Line Search: 1.4652
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 626a6f00 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.156e+03, gradient 2.534e-01, dot -0.171; [c7279a36-9984-413a-a520-a8c3c0d3a440 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -60.51469948895699, -25.740148195871996, -3.736914585272876, 8.134354798372286
LBFGS Accumulation History: 3 points
Removed measurement 626a6f00 to history. Total: 3
th(0)=-60.51469948895699;dx=-0.06397299749124813
New Minimum: -60.51469948895699 > -94.87095463688061
END: th(15511.929768229566)=-94.87095463688061; dx=-0.00163871604960226 evalInputDelta=34.35625514792362
Fitness changed from -60.51469948895699 to -94.87095463688061
Iteration 4 complete. Error: -94.87095463688061 Total: 2.8717; Orientation: 0.3015; Line Search: 1.7941
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 8bf3147 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.952e+03, gradient 2.464e-01, dot -0.119; [c7279a36-9984-413a-a520-a8c3c0d3a440 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -94.87095463688061, -25.740148195871996, -3.736914585272876, 8.134354798372286
LBFGS Accumulation History: 3 points
Removed measurement 8bf3147 to history. Total: 3
th(0)=-94.87095463688061;dx=-0.060538950263121186
New Minimum: -94.87095463688061 > -154.93793003518934
END: th(33419.43960201202)=-154.93793003518934; dx=-6.180754163600986E-4 evalInputDelta=60.06697539830873
Fitness changed from -94.87095463688061 to -154.93793003518934
Iteration 5 complete. Error: -154.93793003518934 Total: 2.6042; Orientation: 0.3208; Line Search: 1.5908
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 293494e1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.729e+03, gradient 2.089e-01, dot -0.100; [c7279a36-9984-413a-a520-a8c3c0d3a440 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -154.93793003518934, -25.740148195871996, -3.736914585272876, 8.134354798372286
LBFGS Accumulation History: 3 points
Removed measurement 293494e1 to history. Total: 3
th(0)=-154.93793003518934;dx=-0.043289234892443046
New Minimum: -154.93793003518934 > -166.79278119156572
END: th(72000.00000000003)=-166.79278119156572; dx=-4.167996730637012E-4 evalInputDelta=11.854851156376384
Fitness changed from -154.93793003518934 to -166.79278119156572
Iteration 6 complete. Error: -166.79278119156572 Total: 2.9121; Orientation: 0.3932; Line Search: 1.5916
Adding measurement 60f0dbb0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.097e+03, gradient 2.490e-01, dot -0.044; [c7279a36-9984-413a-a520-a8c3c0d3a440 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -166.79278119156572, -25.740148195871996, -3.736914585272876, 8.134354798372286
LBFGS Accumulation History: 3 points
Removed measurement 60f0dbb0 to history. Total: 3
th(0)=-166.79278119156572;dx=-0.06166416445836812
New Minimum: -166.79278119156572 > -204.7213804486354
WOLF (strong): th(155119.2976822957)=-204.7213804486354; dx=5.897154471286138E-5 evalInputDelta=37.92859925706966
New Minimum: -204.7213804486354 > -303.8797959731908
END: th(77559.64884114785)=-303.8797959731908; dx=-8.583118807575308E-4 evalInputDelta=137.0870147816251
Fitness changed from -166.79278119156572 to -303.8797959731908
Iteration 7 complete. Error: -303.8797959731908 Total: 4.0042; Orientation: 0.2996; Line Search: 2.9987
Adding measurement 402ab034 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.390e+04, gradient 1.945e-01, dot -0.036; [c7279a36-9984-413a-a520-a8c3c0d3a440 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -303.8797959731908, -25.740148195871996, -3.736914585272876, 8.134354798372286
LBFGS Accumulation History: 3 points
Removed measurement 402ab034 to history. Total: 3
th(0)=-303.8797959731908;dx=-0.037805085157456814
Armijo: th(167097.19801006012)=-254.8684966199216; dx=-1.714421688910702E-4 evalInputDelta=-49.01129935326924
New Minimum: -303.8797959731908 > -401.59295304715846
END: th(83548.59900503006)=-401.59295304715846; dx=-0.0013177066632772108 evalInputDelta=97.71315707396764
Fitness changed from -303.8797959731908 to -401.59295304715846
Iteration 8 complete. Error: -401.59295304715846 Total: 3.3810; Orientation: 0.2844; Line Search: 2.4224
Adding measurement 2d57a197 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.011e+05, gradient 2.116e-01, dot -0.040; [c7279a36-9984-413a-a520-a8c3c0d3a440 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -401.59295304715846, -25.740148195871996, -3.736914585272876, 8.134354798372286
LBFGS Accumulation History: 3 points
Removed measurement 2d57a197 to history. Total: 3
th(0)=-401.59295304715846;dx=-0.04471489501446552
Armijo: th(180000.0000000001)=-299.2146639269106; dx=4.333939546347045E-4 evalInputDelta=-102.37828912024787
New Minimum: -401.59295304715846 > -560.7398133088732
END: th(90000.00000000004)=-560.7398133088732; dx=-0.0013866523860312397 evalInputDelta=159.14686026171472
Fitness changed from -401.59295304715846 to -560.7398133088732
Iteration 9 complete. Error: -560.7398133088732 Total: 3.6191; Orientation: 0.3078; Line Search: 2.5453
Adding measurement 65c751cb to history. Total: 3
Accepted: LBFGS Orientation magnitude: 8.977e+03, gradient 1.931e-01, dot 0.090; [c7279a36-9984-413a-a520-a8c3c0d3a440 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-560.7398133088732;dx=-155.30920816517204
Armijo: th(193899.12210286965)=86.0289413385305; dx=0.002406293573555828 evalInputDelta=-646.7687546474037
Armijo: th(96949.56105143482)=86.01959654865578; dx=0.004821218710953782 evalInputDelta=-646.7594098575289
Armijo: th(32316.520350478273)=85.78320941746586; dx=0.014430676017567399 evalInputDelta=-646.523022726339
Armijo: th(8079.130087619568)=84.12225824584561; dx=0.05753186274858036 evalInputDelta=-644.8620715547188
Armijo: th(1615.8260175239136)=73.06070812003713; dx=0.27443042519677663 evalInputDelta=-633.8005214289103
Armijo: th(269.3043362539856)=8.512265500065155; dx=1.2511800858377986 evalInputDelta=-569.2520788089383
Armijo: th(38.47204803628365)=-108.34109261144255; dx=5.5309867869493035 evalInputDelta=-452.39872069743063
Armijo: th(4.809006004535457)=-396.11309924034924; dx=6.732567253504575 evalInputDelta=-164.62671406852394
Armijo: th(0.5343340005039396)=-558.3821584611077; dx=-41.18148537186774 evalInputDelta=-2.3576548477655024
New Minimum: -560.7398133088732 > -561.3669073011057
END: th(0.053433400050393966)=-561.3669073011057; dx=-59.794948287989875 evalInputDelta=0.6270939922325169
Fitness changed from -560.7398133088732 to -561.3669073011057
Iteration 10 complete. Error: -561.3669073011057 Total: 9.5524; Orientation: 0.2721; Line Search: 8.6059
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 66fc2635 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 2.616e+08, gradient 1.935e-01, dot -0.046; [c7279a36-9984-413a-a520-a8c3c0d3a440 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -561.3669073011057, -560.7398133088732, -25.740148195871996, -3.736914585272876, 8.134354798372286
Rejected: LBFGS Orientation magnitude: 6.094e+08, gradient 1.935e-01, dot -0.045; [c7279a36-9984-413a-a520-a8c3c0d3a440 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -561.3669073011057, -560.7398133088732, -25.740148195871996, -3.736914585272876
LBFGS Accumulation History: 3 points
Removed measurement 66fc2635 to history. Total: 4
Removed measurement 65c751cb to history. Total: 3
th(0)=-561.3669073011057;dx=-0.037333299563006565
New Minimum: -561.3669073011057 > -561.36951918603
END: th(0.11511877067492017)=-561.36951918603; dx=-0.03247318683608727 evalInputDelta=0.0026118849242493525
Fitness changed from -561.3669073011057 to -561.36951918603
Iteration 11 complete. Error: -561.36951918603 Total: 3.0264; Orientation: 0.7920; Line Search: 1.5207
Adding measurement a89411 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 2.246e+04, gradient 1.935e-01, dot 0.061; [c7279a36-9984-413a-a520-a8c3c0d3a440 = 1.000/1.000e+00]
th(0)=-561.36951918603;dx=-261.0678365790317
Armijo: th(0.24801587301587316)=-543.1257680447725; dx=-15.091374557639893 evalInputDelta=-18.243751141257462
Armijo: th(0.12400793650793658)=-554.8222813247481; dx=-31.915532534734037 evalInputDelta=-6.547237861281815
Armijo: th(0.04133597883597886)=-559.8411226697107; dx=-44.42165704368969 evalInputDelta=-1.528396516319276
Armijo: th(0.010333994708994714)=-561.0640881341794; dx=-52.036565295941855 evalInputDelta=-0.305431051850519
Armijo: th(0.002066798941798943)=-561.3129093573065; dx=-53.94923538952342 evalInputDelta=-0.05660982872348086
Armijo: th(3.4446649029982383E-4)=-561.3603550370321; dx=-53.497058968864465 evalInputDelta=-0.009164148997797383
Armijo: th(4.9209498614260546E-5)=-561.3681499580814; dx=-49.346330074744955 evalInputDelta=-0.001369227948543994
Armijo: th(6.151187326782568E-6)=-561.3692678859408; dx=-10.511362793140526 evalInputDelta=-2.51300089189499E-4
Armijo: th(6.834652585313965E-7)=-561.3695063045199; dx=109.4612543711913 evalInputDelta=-1.2881510087936476E-5
New Minimum: -561.36951918603 > -561.3695349147494
WOLF (strong): th(6.834652585313965E-8)=-561.3695349147494; dx=132.19134052371692 evalInputDelta=1.5728719404251024E-5
New Minimum: -561.3695349147494 > -561.3695444514925
WOLF (strong): th(6.2133205321036046E-9)=-561.3695444514925; dx=124.19438846122083 evalInputDelta=2.5265462568313524E-5
WOLF (strong): th(5.177767110086337E-10)=-561.3695444514925; dx=124.04234133799564 evalInputDelta=2.5265462568313524E-5
MIN ALPHA (3.98289777698949E-11): th(6.2133205321036046E-9)=-561.3695444514925
Fitness changed from -561.36951918603 to -561.3695444514925
Iteration 12 complete. Error: -561.3695444514925 Total: 12.0301; Orientation: 0.2897; Line Search: 11.0215
Adding measurement 1b09aca7 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 3.691e+08, gradient 1.936e-01, dot -0.029; [c7279a36-9984-413a-a520-a8c3c0d3a440 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -561.3695444514925, -561.36951918603, -25.740148195871996, -3.736914585272876, 8.134354798372286
Rejected: LBFGS Orientation magnitude: 8.847e+08, gradient 1.936e-01, dot -0.028; [c7279a36-9984-413a-a520-a8c3c0d3a440 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -561.3695444514925, -561.36951918603, -25.740148195871996, -3.736914585272876
LBFGS Accumulation History: 3 points
Removed measurement 1b09aca7 to history. Total: 4
Removed measurement a89411 to history. Total: 3
th(0)=-561.3695444514925;dx=-0.037346862754476455
END: th(2.1544346900318838E-10)=-561.3695444514925; dx=-0.03248669748244882 evalInputDelta=0.0
Fitness changed from -561.3695444514925 to -561.3695444514925
Static Iteration Total: 2.8354; Orientation: 0.6188; Line Search: 1.4838
Iteration 13 failed. Error: -561.3695444514925
Previous Error: 0.0 -> -561.3695444514925
Retrying iteration 13
Reset training subject: 210686230796100
Adding measurement 7c3c21f7 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-561.3695444514925;dx=-0.037378101755815255
END: th(4.6415888336127794E-10)=-561.3695444514925; dx=-0.03246297933100086 evalInputDelta=0.0
Fitness changed from -561.3695444514925 to -561.3695444514925
Static Iteration Total: 3.0906; Orientation: 0.0155; Line Search: 1.5446
Iteration 14 failed. Error: -561.3695444514925
Previous Error: 0.0 -> -561.3695444514925
Optimization terminated 14
Final threshold in iteration 14: -561.3695444514925 (> -Infinity) after 62.254s (< 5400.000s)
Returns
-561.3695444514925