BasicOptimizer.scala:89 executed in 45.76 seconds (0.783 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 222316297967600
Reset training subject: 222316939993200
Adding measurement 2eb4fbce to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-117.8840300093698;dx=-0.08668339337838665
New Minimum: -117.8840300093698 > -118.01272751128056
WOLFE (weak): th(2.154434690031884)=-118.01272751128056; dx=-0.08592039748601496 evalInputDelta=0.12869750191076434
New Minimum: -118.01272751128056 > -118.14229696500816
WOLFE (weak): th(4.308869380063768)=-118.14229696500816; dx=-0.08600661663886286 evalInputDelta=0.2582669556383621
New Minimum: -118.14229696500816 > -118.66345152137919
WOLFE (weak): th(12.926608140191302)=-118.66345152137919; dx=-0.08623100663937214 evalInputDelta=0.7794215120093924
New Minimum: -118.66345152137919 > -121.09612403740226
WOLFE (weak): th(51.70643256076521)=-121.09612403740226; dx=-0.08592438463146014 evalInputDelta=3.2120940280324675
New Minimum: -121.09612403740226 > -134.33238739087147
END: th(258.53216280382605)=-134.33238739087147; dx=-0.07797563934208479 evalInputDelta=16.44835738150168
Fitness changed from -117.8840300093698 to -134.33238739087147
Iteration 1 complete. Error: -134.33238739087147 Total: 5.5178; Orientation: 0.0157; Line Search: 3.7153
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 3403aa73 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-134.33238739087147;dx=-0.089632067836228
New Minimum: -134.33238739087147 > -168.55078250122222
END: th(556.9906600335335)=-168.55078250122222; dx=-0.07195862030528069 evalInputDelta=34.21839511035074
Fitness changed from -134.33238739087147 to -168.55078250122222
Iteration 2 complete. Error: -168.55078250122222 Total: 2.0237; Orientation: 0.0160; Line Search: 1.4069
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 6d417caa to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-168.55078250122222;dx=-0.0955402115302248
New Minimum: -168.55078250122222 > -231.97546011254005
END: th(1200.0)=-231.97546011254005; dx=-0.05652360054951373 evalInputDelta=63.42467761131783
Fitness changed from -168.55078250122222 to -231.97546011254005
Iteration 3 complete. Error: -231.97546011254005 Total: 1.8418; Orientation: 0.0166; Line Search: 1.2200
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement d5883e0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.313e+04, gradient 3.140e-01, dot -0.529; [2b5ce0ca-a5e3-4a97-b661-e0c114032e2a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -231.97546011254005, -168.55078250122222, -134.33238739087147, -117.8840300093698
LBFGS Accumulation History: 3 points
Removed measurement d5883e0 to history. Total: 3
th(0)=-231.97546011254005;dx=-0.0984400770250649
New Minimum: -231.97546011254005 > -304.25861118148464
END: th(2585.3216280382608)=-304.25861118148464; dx=-0.03666204030815479 evalInputDelta=72.28315106894459
Fitness changed from -231.97546011254005 to -304.25861118148464
Iteration 4 complete. Error: -304.25861118148464 Total: 2.1477; Orientation: 0.3049; Line Search: 1.2483
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 4178838c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.208e+04, gradient 3.287e-01, dot -0.318; [2b5ce0ca-a5e3-4a97-b661-e0c114032e2a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -304.25861118148464, -168.55078250122222, -134.33238739087147, -117.8840300093698
LBFGS Accumulation History: 3 points
Removed measurement 4178838c to history. Total: 3
th(0)=-304.25861118148464;dx=-0.10793724612862002
New Minimum: -304.25861118148464 > -365.03378788008996
END: th(5569.9066003353355)=-365.03378788008996; dx=-0.017116608813289844 evalInputDelta=60.77517669860532
Fitness changed from -304.25861118148464 to -365.03378788008996
Iteration 5 complete. Error: -365.03378788008996 Total: 2.1854; Orientation: 0.3123; Line Search: 1.2779
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 183c6a89 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.329e+03, gradient 3.306e-01, dot -0.181; [2b5ce0ca-a5e3-4a97-b661-e0c114032e2a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -365.03378788008996, -168.55078250122222, -134.33238739087147, -117.8840300093698
LBFGS Accumulation History: 3 points
Removed measurement 183c6a89 to history. Total: 3
th(0)=-365.03378788008996;dx=-0.10927456133283397
New Minimum: -365.03378788008996 > -377.0691229171225
END: th(12000.000000000002)=-377.0691229171225; dx=-0.008329267347536261 evalInputDelta=12.035335037032553
Fitness changed from -365.03378788008996 to -377.0691229171225
Iteration 6 complete. Error: -377.0691229171225 Total: 2.1610; Orientation: 0.3251; Line Search: 1.2499
Adding measurement 67357242 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.668e+03, gradient 3.163e-01, dot -0.114; [2b5ce0ca-a5e3-4a97-b661-e0c114032e2a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -377.0691229171225, -168.55078250122222, -134.33238739087147, -117.8840300093698
LBFGS Accumulation History: 3 points
Removed measurement 67357242 to history. Total: 3
th(0)=-377.0691229171225;dx=-0.09988511443230037
Armijo: th(25853.21628038261)=-317.9477611468795; dx=-0.0016713171149313148 evalInputDelta=-59.12136177024303
New Minimum: -377.0691229171225 > -485.8384588224697
END: th(12926.608140191305)=-485.8384588224697; dx=-0.009111811758714375 evalInputDelta=108.76933590534719
Fitness changed from -377.0691229171225 to -485.8384588224697
Iteration 7 complete. Error: -485.8384588224697 Total: 3.0579; Orientation: 0.3212; Line Search: 2.1639
Adding measurement 452f9303 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.548e+03, gradient 2.857e-01, dot -0.116; [2b5ce0ca-a5e3-4a97-b661-e0c114032e2a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -485.8384588224697, -168.55078250122222, -134.33238739087147, -117.8840300093698
LBFGS Accumulation History: 3 points
Removed measurement 452f9303 to history. Total: 3
th(0)=-485.8384588224697;dx=-0.08151220419639821
Armijo: th(27849.533001676682)=-451.4980805855763; dx=-0.0015193930565940805 evalInputDelta=-34.3403782368934
New Minimum: -485.8384588224697 > -657.2091174704248
END: th(13924.766500838341)=-657.2091174704248; dx=-0.013356340602193501 evalInputDelta=171.37065864795505
Fitness changed from -485.8384588224697 to -657.2091174704248
Iteration 8 complete. Error: -657.2091174704248 Total: 2.8023; Orientation: 0.3147; Line Search: 1.9174
Adding measurement 55c88fe7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.488e+04, gradient 2.848e-01, dot -0.137; [2b5ce0ca-a5e3-4a97-b661-e0c114032e2a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -657.2091174704248, -168.55078250122222, -134.33238739087147, -117.8840300093698
LBFGS Accumulation History: 3 points
Removed measurement 55c88fe7 to history. Total: 3
th(0)=-657.2091174704248;dx=-0.08099365126755607
Armijo: th(30000.00000000001)=-563.7333783695982; dx=-0.001724804678112093 evalInputDelta=-93.47573910082656
New Minimum: -657.2091174704248 > -832.5878336182149
END: th(15000.000000000005)=-832.5878336182149; dx=-0.015443709204715366 evalInputDelta=175.3787161477901
Fitness changed from -657.2091174704248 to -832.5878336182149
Iteration 9 complete. Error: -832.5878336182149 Total: 2.8937; Orientation: 0.3207; Line Search: 1.9599
Adding measurement 4ea2f83 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.572e+04, gradient 2.888e-01, dot -0.157; [2b5ce0ca-a5e3-4a97-b661-e0c114032e2a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -832.5878336182149, -168.55078250122222, -134.33238739087147, -117.8840300093698
LBFGS Accumulation History: 3 points
Removed measurement 4ea2f83 to history. Total: 3
th(0)=-832.5878336182149;dx=-0.08334226788467608
Armijo: th(32316.52035047827)=-664.1376852026917; dx=-9.122808190445129E-4 evalInputDelta=-168.45014841552313
New Minimum: -832.5878336182149 > -998.5228550475772
END: th(16158.260175239135)=-998.5228550475772; dx=-0.015101419930569737 evalInputDelta=165.93502142936234
Fitness changed from -832.5878336182149 to -998.5228550475772
Iteration 10 complete. Error: -998.5228550475772 Total: 2.8049; Orientation: 0.3293; Line Search: 1.9027
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 1b9826ab to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.987e+05, gradient 3.011e-01, dot -0.180; [2b5ce0ca-a5e3-4a97-b661-e0c114032e2a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -998.5228550475772, -168.55078250122222, -134.33238739087147, -117.8840300093698
LBFGS Accumulation History: 3 points
Removed measurement 1b9826ab to history. Total: 3
th(0)=-998.5228550475772;dx=-0.09063362733000958
Armijo: th(34811.916252095856)=-747.5857844459446; dx=0.0015452765329467787 evalInputDelta=-250.93707060163263
New Minimum: -998.5228550475772 > -1154.4748518865833
END: th(17405.958126047928)=-1154.4748518865833; dx=-0.013290372302298202 evalInputDelta=155.95199683900614
Fitness changed from -998.5228550475772 to -1154.4748518865833
Iteration 11 complete. Error: -1154.4748518865833 Total: 2.7223; Orientation: 0.2966; Line Search: 1.8183
Adding measurement 31badf54 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 9.182e+04, gradient 3.107e-01, dot 0.187; [2b5ce0ca-a5e3-4a97-b661-e0c114032e2a = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-1154.4748518865833;dx=-5330.699170027153
Armijo: th(37500.000000000015)=363.29524698837355; dx=0.019063098589744483 evalInputDelta=-1517.7700988749568
Armijo: th(18750.000000000007)=360.8090852101361; dx=0.03809399381253739 evalInputDelta=-1515.2839370967195
Armijo: th(6250.000000000003)=354.50840003454437; dx=0.1137334238680826 evalInputDelta=-1508.9832519211277
Armijo: th(1562.5000000000007)=341.4776618752373; dx=0.44743377439331833 evalInputDelta=-1495.9525137618207
Armijo: th(312.5000000000001)=309.5061955102222; dx=2.133866091047812 evalInputDelta=-1463.9810473968055
Armijo: th(52.08333333333335)=246.86413725730392; dx=11.782249375251357 evalInputDelta=-1401.3389891438874
Armijo: th(7.4404761904761925)=111.83865618364587; dx=68.10347911589636 evalInputDelta=-1266.313508070229
Armijo: th(0.9300595238095241)=-306.9017399568371; dx=262.8278702506514 evalInputDelta=-847.5731119297462
Armijo: th(0.10333994708994712)=-1079.5230024021978; dx=-571.7651419194481 evalInputDelta=-74.95184948438555
New Minimum: -1154.4748518865833 > -1170.3298460373737
END: th(0.010333994708994713)=-1170.3298460373737; dx=-3037.2935633798015 evalInputDelta=15.85499415079039
Fitness changed from -1154.4748518865833 to -1170.3298460373737
Iteration 12 complete. Error: -1170.3298460373737 Total: 8.2214; Orientation: 0.2767; Line Search: 7.0134
Adding measurement 3a3b9b1c to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.629e+06, gradient 3.079e-01, dot -0.168; [2b5ce0ca-a5e3-4a97-b661-e0c114032e2a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1170.3298460373737, -1154.4748518865833, -168.55078250122222, -134.33238739087147, -117.8840300093698
Rejected: LBFGS Orientation magnitude: 1.701e+06, gradient 3.079e-01, dot -0.189; [2b5ce0ca-a5e3-4a97-b661-e0c114032e2a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1170.3298460373737, -1154.4748518865833, -168.55078250122222, -134.33238739087147
LBFGS Accumulation History: 3 points
Removed measurement 3a3b9b1c to history. Total: 4
Removed measurement 31badf54 to history. Total: 3
th(0)=-1170.3298460373737;dx=-0.09475661974504943
New Minimum: -1170.3298460373737 > -1170.3309922863473
END: th(0.022263916687664153)=-1170.3309922863473; dx=-0.08302673371756891 evalInputDelta=0.0011462489735549752
Fitness changed from -1170.3298460373737 to -1170.3309922863473
Iteration 13 complete. Error: -1170.3309922863473 Total: 2.6644; Orientation: 0.6729; Line Search: 1.3863
Adding measurement 6a4223cf to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.069e+05, gradient 3.079e-01, dot -0.170; [2b5ce0ca-a5e3-4a97-b661-e0c114032e2a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1170.3309922863473, -168.55078250122222, -134.33238739087147, -117.8840300093698
LBFGS Accumulation History: 3 points
Removed measurement 6a4223cf to history. Total: 3
th(0)=-1170.3309922863473;dx=-0.09475089011536508
New Minimum: -1170.3309922863473 > -1170.3333794484643
END: th(0.04796615444788341)=-1170.3333794484643; dx=-0.08305814291790842 evalInputDelta=0.0023871621169746504
Fitness changed from -1170.3309922863473 to -1170.3333794484643
Iteration 14 complete. Error: -1170.3333794484643 Total: 2.4477; Orientation: 0.5339; Line Search: 1.3375
Adding measurement ffe2c8a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.025e+05, gradient 3.079e-01, dot -0.170; [2b5ce0ca-a5e3-4a97-b661-e0c114032e2a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1170.3333794484643, -168.55078250122222, -134.33238739087147, -117.8840300093698
LBFGS Accumulation History: 3 points
Removed measurement ffe2c8a to history. Total: 3
th(0)=-1170.3333794484643;dx=-0.0947490838888351
New Minimum: -1170.3333794484643 > -1170.3384190361205
END: th(0.10333994708994715)=-1170.3384190361205; dx=-0.08304976504668055 evalInputDelta=0.005039587656256117
Fitness changed from -1170.3333794484643 to -1170.3384190361205
Iteration 15 complete. Error: -1170.3384190361205 Total: 2.2460; Orientation: 0.2961; Line Search: 1.3441
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -1170.3384190361205 (> -Infinity) after 45.759s (< 5400.000s)
Returns
-1170.3384190361205