BasicOptimizer.scala:89 executed in 54.12 seconds (1.565 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 210176337992400
Reset training subject: 210177138929600
Adding measurement 63ad4eec to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-2.5640275297928383;dx=-0.022481768710375577
New Minimum: -2.5640275297928383 > -2.6047029099763996
WOLFE (weak): th(2.154434690031884)=-2.6047029099763996; dx=-0.022489930927054806 evalInputDelta=0.040675380183561316
New Minimum: -2.6047029099763996 > -2.6452976828423207
WOLFE (weak): th(4.308869380063768)=-2.6452976828423207; dx=-0.022397258461270768 evalInputDelta=0.08127015304948237
New Minimum: -2.6452976828423207 > -2.808844071654228
WOLFE (weak): th(12.926608140191302)=-2.808844071654228; dx=-0.022599856502518695 evalInputDelta=0.24481654186138968
New Minimum: -2.808844071654228 > -3.568593386555497
WOLFE (weak): th(51.70643256076521)=-3.568593386555497; dx=-0.023790704942248823 evalInputDelta=1.0045658567626585
New Minimum: -3.568593386555497 > -8.071274383393668
WOLFE (weak): th(258.53216280382605)=-8.071274383393668; dx=-0.026918889554546275 evalInputDelta=5.507246853600829
New Minimum: -8.071274383393668 > -35.02159577508951
END: th(1551.1929768229563)=-35.02159577508951; dx=-0.016276108247653985 evalInputDelta=32.45756824529667
Fitness changed from -2.5640275297928383 to -35.02159577508951
Iteration 1 complete. Error: -35.02159577508951 Total: 7.5230; Orientation: 0.0156; Line Search: 5.3634
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 73e0591a to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-35.02159577508951;dx=-0.04518634001310827
New Minimum: -35.02159577508951 > -84.68744597475533
END: th(3341.943960201201)=-84.68744597475533; dx=-0.010123744617258179 evalInputDelta=49.665850199665826
Fitness changed from -35.02159577508951 to -84.68744597475533
Iteration 2 complete. Error: -84.68744597475533 Total: 2.2455; Orientation: 0.0158; Line Search: 1.5153
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 2567a2e8 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-84.68744597475533;dx=-0.027741020120315103
New Minimum: -84.68744597475533 > -181.6888393297812
END: th(7200.000000000001)=-181.6888393297812; dx=-0.004997838030385211 evalInputDelta=97.00139335502585
Fitness changed from -84.68744597475533 to -181.6888393297812
Iteration 3 complete. Error: -181.6888393297812 Total: 2.3884; Orientation: 0.0155; Line Search: 1.6873
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 57011736 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.289e+03, gradient 1.618e-01, dot -0.191; [d3b5c953-fba0-46c5-bd6e-3f53967d7d5f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -181.6888393297812, -84.68744597475533, -35.02159577508951, -2.5640275297928383
LBFGS Accumulation History: 3 points
Removed measurement 57011736 to history. Total: 3
th(0)=-181.6888393297812;dx=-0.026048306685495147
New Minimum: -181.6888393297812 > -290.10923293322804
END: th(15511.929768229566)=-290.10923293322804; dx=-0.0021849324019210994 evalInputDelta=108.42039360344685
Fitness changed from -181.6888393297812 to -290.10923293322804
Iteration 4 complete. Error: -290.10923293322804 Total: 2.5372; Orientation: 0.2916; Line Search: 1.5065
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 1ecda867 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.254e+03, gradient 1.766e-01, dot -0.144; [d3b5c953-fba0-46c5-bd6e-3f53967d7d5f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -290.10923293322804, -84.68744597475533, -35.02159577508951, -2.5640275297928383
LBFGS Accumulation History: 3 points
Removed measurement 1ecda867 to history. Total: 3
th(0)=-290.10923293322804;dx=-0.031119731729865843
New Minimum: -290.10923293322804 > -333.4706188450921
END: th(33419.43960201202)=-333.4706188450921; dx=-1.995586301865104E-4 evalInputDelta=43.36138591186403
Fitness changed from -290.10923293322804 to -333.4706188450921
Iteration 5 complete. Error: -333.4706188450921 Total: 2.6101; Orientation: 0.3071; Line Search: 1.5325
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 3be6be5d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.342e+03, gradient 1.842e-01, dot -0.113; [d3b5c953-fba0-46c5-bd6e-3f53967d7d5f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -333.4706188450921, -84.68744597475533, -35.02159577508951, -2.5640275297928383
LBFGS Accumulation History: 3 points
Removed measurement 3be6be5d to history. Total: 3
th(0)=-333.4706188450921;dx=-0.03388067937470436
Armijo: th(72000.00000000003)=-322.67888620572137; dx=6.91655063505712E-4 evalInputDelta=-10.791732639370707
New Minimum: -333.4706188450921 > -482.0478535308545
END: th(36000.000000000015)=-482.0478535308545; dx=-1.1443519928584423E-4 evalInputDelta=148.57723468576245
Fitness changed from -333.4706188450921 to -482.0478535308545
Iteration 6 complete. Error: -482.0478535308545 Total: 3.5477; Orientation: 0.2926; Line Search: 2.5107
Adding measurement 2fdafc72 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.084e+04, gradient 1.728e-01, dot -0.109; [d3b5c953-fba0-46c5-bd6e-3f53967d7d5f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -482.0478535308545, -84.68744597475533, -35.02159577508951, -2.5640275297928383
LBFGS Accumulation History: 3 points
Removed measurement 2fdafc72 to history. Total: 3
th(0)=-482.0478535308545;dx=-0.029721972135195368
Armijo: th(77559.64884114785)=-431.08691876634174; dx=0.0010243470315766065 evalInputDelta=-50.960934764512785
New Minimum: -482.0478535308545 > -636.5107145263203
END: th(38779.824420573925)=-636.5107145263203; dx=-8.857687685157093E-4 evalInputDelta=154.46286099546575
Fitness changed from -482.0478535308545 to -636.5107145263203
Iteration 7 complete. Error: -636.5107145263203 Total: 3.2714; Orientation: 0.3141; Line Search: 2.2444
Adding measurement 550f7299 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.380e+04, gradient 1.620e-01, dot -0.121; [d3b5c953-fba0-46c5-bd6e-3f53967d7d5f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -636.5107145263203, -84.68744597475533, -35.02159577508951, -2.5640275297928383
LBFGS Accumulation History: 3 points
Removed measurement 550f7299 to history. Total: 3
th(0)=-636.5107145263203;dx=-0.026180321869838236
Armijo: th(83548.59900503006)=-548.4760805679966; dx=0.0016227931273934803 evalInputDelta=-88.03463395832364
New Minimum: -636.5107145263203 > -822.0901555931865
END: th(41774.29950251503)=-822.0901555931865; dx=-7.498073224990742E-4 evalInputDelta=185.57944106686625
Fitness changed from -636.5107145263203 to -822.0901555931865
Iteration 8 complete. Error: -822.0901555931865 Total: 3.5381; Orientation: 0.3055; Line Search: 2.4756
Adding measurement 15c42753 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.511e+04, gradient 1.496e-01, dot -0.147; [d3b5c953-fba0-46c5-bd6e-3f53967d7d5f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -822.0901555931865, -84.68744597475533, -35.02159577508951, -2.5640275297928383
LBFGS Accumulation History: 3 points
Removed measurement 15c42753 to history. Total: 3
th(0)=-822.0901555931865;dx=-0.022376093018368577
Armijo: th(90000.00000000004)=-681.0137066639712; dx=0.0016041395370064973 evalInputDelta=-141.0764489292153
New Minimum: -822.0901555931865 > -969.6418176051059
END: th(45000.00000000002)=-969.6418176051059; dx=-8.004697865845588E-4 evalInputDelta=147.55166201191935
Fitness changed from -822.0901555931865 to -969.6418176051059
Iteration 9 complete. Error: -969.6418176051059 Total: 3.2448; Orientation: 0.2800; Line Search: 2.2884
Adding measurement 6a1fcfa0 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 8.545e+04, gradient 1.538e-01, dot 0.151; [d3b5c953-fba0-46c5-bd6e-3f53967d7d5f = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-969.6418176051059;dx=-1982.1016985632873
Armijo: th(96949.56105143482)=-331.22885656888025; dx=0.0079195354425542 evalInputDelta=-638.4129610362256
Armijo: th(48474.78052571741)=-331.26217597353457; dx=0.01585999189896386 evalInputDelta=-638.3796416315713
Armijo: th(16158.260175239137)=-331.3725139302635; dx=0.04758380231743194 evalInputDelta=-638.2693036748424
Armijo: th(4039.565043809784)=-331.56180027198866; dx=0.19046604599084072 evalInputDelta=-638.0800173331172
Armijo: th(807.9130087619568)=-334.67030678280344; dx=0.9466444680769337 evalInputDelta=-634.9715108223024
Armijo: th(134.6521681269928)=-364.65206477504694; dx=5.2633807023722365 evalInputDelta=-604.9897528300589
Armijo: th(19.236024018141826)=-536.3738223392882; dx=22.004349928999495 evalInputDelta=-433.2679952658177
Armijo: th(2.4045030022677283)=-624.4059863604834; dx=101.5154912160057 evalInputDelta=-345.23583124462243
Armijo: th(0.2671670002519698)=-874.4653376021915; dx=260.0288359517588 evalInputDelta=-95.17648000291433
New Minimum: -969.6418176051059 > -988.0149134560867
END: th(0.026716700025196983)=-988.0149134560867; dx=-371.10997054380834 evalInputDelta=18.373095850980803
Fitness changed from -969.6418176051059 to -988.0149134560867
Iteration 10 complete. Error: -988.0149134560867 Total: 9.8562; Orientation: 0.2991; Line Search: 8.8790
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 205b56a to history. Total: 4
Rejected: LBFGS Orientation magnitude: 2.123e+06, gradient 1.509e-01, dot -0.134; [d3b5c953-fba0-46c5-bd6e-3f53967d7d5f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -988.0149134560867, -969.6418176051059, -84.68744597475533, -35.02159577508951, -2.5640275297928383
Rejected: LBFGS Orientation magnitude: 4.743e+06, gradient 1.509e-01, dot -0.152; [d3b5c953-fba0-46c5-bd6e-3f53967d7d5f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -988.0149134560867, -969.6418176051059, -84.68744597475533, -35.02159577508951
LBFGS Accumulation History: 3 points
Removed measurement 205b56a to history. Total: 4
Removed measurement 6a1fcfa0 to history. Total: 3
th(0)=-988.0149134560867;dx=-0.022757822255520352
New Minimum: -988.0149134560867 > -988.0159840974625
END: th(0.057559385337460085)=-988.0159840974625; dx=-0.01717927820282073 evalInputDelta=0.001070641375804371
Fitness changed from -988.0149134560867 to -988.0159840974625
Iteration 11 complete. Error: -988.0159840974625 Total: 3.0753; Orientation: 0.8376; Line Search: 1.5312
Adding measurement 73e49b77 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.933e+05, gradient 1.510e-01, dot -0.134; [d3b5c953-fba0-46c5-bd6e-3f53967d7d5f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -988.0159840974625, -84.68744597475533, -35.02159577508951, -2.5640275297928383
LBFGS Accumulation History: 3 points
Removed measurement 73e49b77 to history. Total: 3
th(0)=-988.0159840974625;dx=-0.022747532301700152
New Minimum: -988.0159840974625 > -988.018166555249
END: th(0.12400793650793658)=-988.018166555249; dx=-0.017182522739411635 evalInputDelta=0.002182457786489067
Fitness changed from -988.0159840974625 to -988.018166555249
Iteration 12 complete. Error: -988.018166555249 Total: 2.4594; Orientation: 0.3080; Line Search: 1.4300
Adding measurement 35c45545 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.354e+05, gradient 1.510e-01, dot -0.134; [d3b5c953-fba0-46c5-bd6e-3f53967d7d5f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -988.018166555249, -84.68744597475533, -35.02159577508951, -2.5640275297928383
LBFGS Accumulation History: 3 points
Removed measurement 35c45545 to history. Total: 3
th(0)=-988.018166555249;dx=-0.022775717847334523
New Minimum: -988.018166555249 > -988.0229891856015
END: th(0.2671670002519699)=-988.0229891856015; dx=-0.017194465249665485 evalInputDelta=0.004822630352578017
Fitness changed from -988.018166555249 to -988.0229891856015
Iteration 13 complete. Error: -988.0229891856015 Total: 2.5544; Orientation: 0.2896; Line Search: 1.5708
Adding measurement 10c383bb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.145e+06, gradient 1.510e-01, dot -0.135; [d3b5c953-fba0-46c5-bd6e-3f53967d7d5f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -988.0229891856015, -84.68744597475533, -35.02159577508951, -2.5640275297928383
LBFGS Accumulation History: 3 points
Removed measurement 10c383bb to history. Total: 3
th(0)=-988.0229891856015;dx=-0.022769278114221185
New Minimum: -988.0229891856015 > -988.0333296324048
END: th(0.5755938533746009)=-988.0333296324048; dx=-0.017204034656560473 evalInputDelta=0.01034044680329771
Fitness changed from -988.0229891856015 to -988.0333296324048
Iteration 14 complete. Error: -988.0333296324048 Total: 2.6644; Orientation: 0.2875; Line Search: 1.4904
Adding measurement 4d67eea4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.434e+05, gradient 1.510e-01, dot -0.134; [d3b5c953-fba0-46c5-bd6e-3f53967d7d5f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -988.0333296324048, -84.68744597475533, -35.02159577508951, -2.5640275297928383
LBFGS Accumulation History: 3 points
Removed measurement 4d67eea4 to history. Total: 3
th(0)=-988.0333296324048;dx=-0.022769905913024524
New Minimum: -988.0333296324048 > -988.0555844718781
END: th(1.2400793650793658)=-988.0555844718781; dx=-0.01718576043879768 evalInputDelta=0.022254839473248467
Fitness changed from -988.0333296324048 to -988.0555844718781
Iteration 15 complete. Error: -988.0555844718781 Total: 2.5803; Orientation: 0.2909; Line Search: 1.5638
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -988.0555844718781 (> -Infinity) after 54.118s (< 5400.000s)
Returns
-988.0555844718781