BasicOptimizer.scala:89 executed in 55.37 seconds (1.434 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 208259305369200
Reset training subject: 208260054060300
Adding measurement 3cc5f502 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.035128890514430466;dx=-0.020942793673059824
New Minimum: -0.035128890514430466 > -0.06746239727858594
WOLFE (weak): th(2.154434690031884)=-0.06746239727858594; dx=-0.02096043221125289 evalInputDelta=0.03233350676415547
New Minimum: -0.06746239727858594 > -0.0998231850664073
WOLFE (weak): th(4.308869380063768)=-0.0998231850664073; dx=-0.020968135750642884 evalInputDelta=0.06469429455197684
New Minimum: -0.0998231850664073 > -0.22997274355791042
WOLFE (weak): th(12.926608140191302)=-0.22997274355791042; dx=-0.02113976571687163 evalInputDelta=0.19484385304347995
New Minimum: -0.22997274355791042 > -0.8250452484982027
WOLFE (weak): th(51.70643256076521)=-0.8250452484982027; dx=-0.021518080764577834 evalInputDelta=0.7899163579837722
New Minimum: -0.8250452484982027 > -4.184727971728265
WOLFE (weak): th(258.53216280382605)=-4.184727971728265; dx=-0.022924321777472553 evalInputDelta=4.149599081213834
New Minimum: -4.184727971728265 > -27.613468096200286
WOLFE (weak): th(1551.1929768229563)=-27.613468096200286; dx=-0.020269895955577762 evalInputDelta=27.578339205685857
New Minimum: -27.613468096200286 > -89.58769717440276
END: th(10858.350837760694)=-89.58769717440276; dx=-0.004863739598946183 evalInputDelta=89.55256828388833
Fitness changed from -0.035128890514430466 to -89.58769717440276
Iteration 1 complete. Error: -89.58769717440276 Total: 8.4472; Orientation: 0.0158; Line Search: 6.2094
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 544a936d to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-89.58769717440276;dx=-0.0310077672010546
New Minimum: -89.58769717440276 > -170.6186241200682
END: th(23393.607721408407)=-170.6186241200682; dx=-0.0033733865955939057 evalInputDelta=81.03092694566544
Fitness changed from -89.58769717440276 to -170.6186241200682
Iteration 2 complete. Error: -170.6186241200682 Total: 2.2983; Orientation: 0.0159; Line Search: 1.5374
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 128ac91a to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-170.6186241200682;dx=-0.03958419043938353
New Minimum: -170.6186241200682 > -192.98974941884333
END: th(50400.00000000001)=-192.98974941884333; dx=-9.952682603840246E-4 evalInputDelta=22.371125298775127
Fitness changed from -170.6186241200682 to -192.98974941884333
Iteration 3 complete. Error: -192.98974941884333 Total: 2.2999; Orientation: 0.0160; Line Search: 1.5288
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 7b23a5cc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.898e+03, gradient 1.893e-01, dot -0.212; [87fa0104-8305-43d9-80ff-d4d4557fb596 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -192.98974941884333, -170.6186241200682, -89.58769717440276, -0.035128890514430466
LBFGS Accumulation History: 3 points
Removed measurement 7b23a5cc to history. Total: 3
th(0)=-192.98974941884333;dx=-0.03579780925976642
New Minimum: -192.98974941884333 > -257.72525675147216
END: th(108583.50837760697)=-257.72525675147216; dx=-5.670352917176637E-4 evalInputDelta=64.73550733262883
Fitness changed from -192.98974941884333 to -257.72525675147216
Iteration 4 complete. Error: -257.72525675147216 Total: 2.7923; Orientation: 0.2958; Line Search: 1.7262
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 62268c77 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.322e+03, gradient 1.607e-01, dot -0.175; [87fa0104-8305-43d9-80ff-d4d4557fb596 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -257.72525675147216, -170.6186241200682, -89.58769717440276, -0.035128890514430466
LBFGS Accumulation History: 3 points
Removed measurement 62268c77 to history. Total: 3
th(0)=-257.72525675147216;dx=-0.025793882427551786
New Minimum: -257.72525675147216 > -264.4137348154255
END: th(233936.07721408413)=-264.4137348154255; dx=-1.617654753108911E-4 evalInputDelta=6.688478063953369
Fitness changed from -257.72525675147216 to -264.4137348154255
Iteration 5 complete. Error: -264.4137348154255 Total: 2.5997; Orientation: 0.3110; Line Search: 1.5407
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 19a26868 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.622e+03, gradient 1.417e-01, dot -0.118; [87fa0104-8305-43d9-80ff-d4d4557fb596 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -264.4137348154255, -170.6186241200682, -89.58769717440276, -0.035128890514430466
LBFGS Accumulation History: 3 points
Removed measurement 19a26868 to history. Total: 3
th(0)=-264.4137348154255;dx=-0.020065680540179147
Armijo: th(504000.0000000002)=-190.8916876620387; dx=-1.1006723398490281E-4 evalInputDelta=-73.52204715338684
New Minimum: -264.4137348154255 > -373.7244353458052
END: th(252000.0000000001)=-373.7244353458052; dx=-7.031205357867485E-4 evalInputDelta=109.31070053037968
Fitness changed from -264.4137348154255 to -373.7244353458052
Iteration 6 complete. Error: -373.7244353458052 Total: 3.3409; Orientation: 0.2914; Line Search: 2.3329
Adding measurement 739834c2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.259e+04, gradient 1.287e-01, dot -0.084; [87fa0104-8305-43d9-80ff-d4d4557fb596 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -373.7244353458052, -170.6186241200682, -89.58769717440276, -0.035128890514430466
LBFGS Accumulation History: 3 points
Removed measurement 739834c2 to history. Total: 3
th(0)=-373.7244353458052;dx=-0.01654344272642047
Armijo: th(542917.5418880349)=-264.4911997268948; dx=-1.2160114615874398E-4 evalInputDelta=-109.23323561891038
New Minimum: -373.7244353458052 > -528.3088791974934
END: th(271458.77094401745)=-528.3088791974934; dx=-8.444695440708036E-4 evalInputDelta=154.5844438516882
Fitness changed from -373.7244353458052 to -528.3088791974934
Iteration 7 complete. Error: -528.3088791974934 Total: 3.5537; Orientation: 0.3028; Line Search: 2.5321
Adding measurement 30853d9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.311e+04, gradient 1.330e-01, dot -0.075; [87fa0104-8305-43d9-80ff-d4d4557fb596 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -528.3088791974934, -170.6186241200682, -89.58769717440276, -0.035128890514430466
LBFGS Accumulation History: 3 points
Removed measurement 30853d9 to history. Total: 3
th(0)=-528.3088791974934;dx=-0.017687938427944423
Armijo: th(584840.1930352104)=-284.9438288717944; dx=6.362059579520814E-5 evalInputDelta=-243.36505032569903
New Minimum: -528.3088791974934 > -731.4313489646296
END: th(292420.0965176052)=-731.4313489646296; dx=-8.840850005081303E-4 evalInputDelta=203.12246976713618
Fitness changed from -528.3088791974934 to -731.4313489646296
Iteration 8 complete. Error: -731.4313489646296 Total: 3.3426; Orientation: 0.2792; Line Search: 2.3409
Adding measurement 4e0b750 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.298e+05, gradient 1.396e-01, dot -0.075; [87fa0104-8305-43d9-80ff-d4d4557fb596 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -731.4313489646296, -170.6186241200682, -89.58769717440276, -0.035128890514430466
LBFGS Accumulation History: 3 points
Removed measurement 4e0b750 to history. Total: 3
th(0)=-731.4313489646296;dx=-0.019466775771665633
Armijo: th(630000.0000000002)=-218.10421589897953; dx=2.559527417535526E-4 evalInputDelta=-513.3271330656501
New Minimum: -731.4313489646296 > -888.5444522149578
END: th(315000.0000000001)=-888.5444522149578; dx=-6.945465213316173E-4 evalInputDelta=157.1131032503282
Fitness changed from -731.4313489646296 to -888.5444522149578
Iteration 9 complete. Error: -888.5444522149578 Total: 3.8087; Orientation: 0.2948; Line Search: 2.7583
Adding measurement 2ccb18c0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.813e+07, gradient 1.521e-01, dot -0.069; [87fa0104-8305-43d9-80ff-d4d4557fb596 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -888.5444522149578, -170.6186241200682, -89.58769717440276, -0.035128890514430466
LBFGS Accumulation History: 3 points
Removed measurement 2ccb18c0 to history. Total: 3
th(0)=-888.5444522149578;dx=-0.023118191498775996
Armijo: th(678646.9273600436)=-187.92676381306018; dx=4.620907456733596E-4 evalInputDelta=-700.6176884018976
New Minimum: -888.5444522149578 > -977.3911585954202
END: th(339323.4636800218)=-977.3911585954202; dx=-2.960377253446038E-4 evalInputDelta=88.84670638046236
Fitness changed from -888.5444522149578 to -977.3911585954202
Iteration 10 complete. Error: -977.3911585954202 Total: 3.2475; Orientation: 0.2770; Line Search: 2.2411
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 6adbab02 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 3.652e+04, gradient 1.544e-01, dot 0.097; [87fa0104-8305-43d9-80ff-d4d4557fb596 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-977.3911585954202;dx=-547.6599956580999
Armijo: th(731050.241294013)=-362.8233710997358; dx=4.5363346668669607E-4 evalInputDelta=-614.5677874956843
Armijo: th(365525.1206470065)=-362.7634638022778; dx=9.069340583183748E-4 evalInputDelta=-614.6276947931424
Armijo: th(121841.7068823355)=-362.74410695102347; dx=0.0027243775144010014 evalInputDelta=-614.6470516443967
Armijo: th(30460.426720583873)=-362.843708477881; dx=0.010889639192586429 evalInputDelta=-614.5474501175391
Armijo: th(6092.085344116775)=-363.0878466511631; dx=0.05481584638334047 evalInputDelta=-614.3033119442571
Armijo: th(1015.3475573527958)=-363.8774065690018; dx=0.3254118530286933 evalInputDelta=-613.5137520264184
Armijo: th(145.0496510503994)=-369.1874806126938; dx=2.25559606441139 evalInputDelta=-608.2036779827264
Armijo: th(18.131206381299926)=-417.81455893325256; dx=16.717637787594985 evalInputDelta=-559.5765996621676
Armijo: th(2.0145784868111027)=-681.2607942099197; dx=65.25724144000257 evalInputDelta=-296.13036438550046
Armijo: th(0.20145784868111027)=-938.4635888051669; dx=-0.969263229873917 evalInputDelta=-38.9275697902533
Armijo: th(0.018314349880100934)=-973.9881279702333; dx=-37.54721632333683 evalInputDelta=-3.4030306251868296
Armijo: th(0.0015261958233417445)=-977.1087933351403; dx=-41.82627450750963 evalInputDelta=-0.28236526027990294
Armijo: th(1.1739967871859574E-4)=-977.3694818494212; dx=-42.40205022575423 evalInputDelta=-0.021676745999002378
Armijo: th(8.385691337042552E-6)=-977.3896155561517; dx=-42.790944925498756 evalInputDelta=-0.0015430392684265826
Armijo: th(5.590460891361701E-7)=-977.3910716922246; dx=-42.494274892260265 evalInputDelta=-8.6903195551713E-5
Armijo: th(3.4940380571010634E-8)=-977.3911098391973; dx=-42.75449877017986 evalInputDelta=-4.8756222895463E-5
Armijo: th(2.0553165041770963E-9)=-977.3911585954202; dx=-42.779348057756714 evalInputDelta=0.0
Armijo: th(1.141842502320609E-10)=-977.3911585954202; dx=-42.228720127179024 evalInputDelta=0.0
MIN ALPHA (6.0096973806347845E-12): th(0.0)=-977.3911585954202
Fitness changed from -977.3911585954202 to -977.3911585954202
Static Iteration Total: 16.7510; Orientation: 0.2947; Line Search: 15.7277
Iteration 11 failed. Error: -977.3911585954202
Previous Error: 0.0 -> -977.3911585954202
Retrying iteration 11
Reset training subject: 208311787285000
Adding measurement 58a43964 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-977.3911585954202;dx=-0.023816155206889865
END: th(2.1544346900318838E-10)=-977.3911585954202; dx=-0.016555204721317607 evalInputDelta=0.0
Fitness changed from -977.3911585954202 to -977.3911585954202
Static Iteration Total: 2.8892; Orientation: 0.0150; Line Search: 1.4824
Iteration 12 failed. Error: -977.3911585954202
Previous Error: 0.0 -> -977.3911585954202
Optimization terminated 12
Final threshold in iteration 12: -977.3911585954202 (> -Infinity) after 55.371s (< 5400.000s)
Returns
-977.3911585954202