BasicOptimizer.scala:89 executed in 7.96 seconds (0.193 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 158023318110400
Reset training subject: 158023447641700
Adding measurement 29dba9bb to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.3807669467139844;dx=-4.227465043591137E-5
New Minimum: -0.3807669467139844 > -0.3808701220476714
WOLFE (weak): th(2.154434690031884)=-0.3808701220476714; dx=-4.190271836911508E-5 evalInputDelta=1.0317533368697784E-4
New Minimum: -0.3808701220476714 > -0.38097394219434616
WOLFE (weak): th(4.308869380063768)=-0.38097394219434616; dx=-4.1405214117254005E-5 evalInputDelta=2.0699548036173798E-4
New Minimum: -0.38097394219434616 > -0.3813895281852608
WOLFE (weak): th(12.926608140191302)=-0.3813895281852608; dx=-4.214461976941472E-5 evalInputDelta=6.225814712763622E-4
New Minimum: -0.3813895281852608 > -0.3832611687923537
WOLFE (weak): th(51.70643256076521)=-0.3832611687923537; dx=-4.212663353526045E-5 evalInputDelta=0.0024942220783693014
New Minimum: -0.3832611687923537 > -0.3934187344851277
WOLFE (weak): th(258.53216280382605)=-0.3934187344851277; dx=-4.2175881193979576E-5 evalInputDelta=0.012651787771143286
New Minimum: -0.3934187344851277 > -0.4594778435367383
WOLFE (weak): th(1551.1929768229563)=-0.4594778435367383; dx=-4.29398451696434E-5 evalInputDelta=0.0787108968227539
New Minimum: -0.4594778435367383 > -0.9082399929040347
END: th(10858.350837760694)=-0.9082399929040347; dx=-3.266981512928588E-5 evalInputDelta=0.5274730461900503
Fitness changed from -0.3807669467139844 to -0.9082399929040347
Iteration 1 complete. Error: -0.9082399929040347 Total: 1.2492; Orientation: 0.0015; Line Search: 0.8961
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 503fd4ea to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-0.9082399929040347;dx=-5.282901556107413E-5
New Minimum: -0.9082399929040347 > -1.7985733481713146
END: th(23393.607721408407)=-1.7985733481713146; dx=-2.1397763738536462E-5 evalInputDelta=0.89033335526728
Fitness changed from -0.9082399929040347 to -1.7985733481713146
Iteration 2 complete. Error: -1.7985733481713146 Total: 0.5629; Orientation: 0.0016; Line Search: 0.2433
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 53bfcfcf to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-1.7985733481713146;dx=-6.299054204769161E-5
New Minimum: -1.7985733481713146 > -1.9072298723481476
WOLF (strong): th(50400.00000000001)=-1.9072298723481476; dx=5.418121090114515E-6 evalInputDelta=0.10865652417683291
New Minimum: -1.9072298723481476 > -2.3820961150364908
END: th(25200.000000000004)=-2.3820961150364908; dx=-5.2307870323774924E-6 evalInputDelta=0.5835227668651761
Fitness changed from -1.7985733481713146 to -2.3820961150364908
Iteration 3 complete. Error: -2.3820961150364908 Total: 0.4577; Orientation: 0.0014; Line Search: 0.3370
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 2c285648 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.377e+03, gradient 8.624e-03, dot -0.334; [1b8511b7-b137-4dac-b07f-4e496baa8372 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.3820961150364908, -1.7985733481713146, -0.9082399929040347, -0.3807669467139844
LBFGS Accumulation History: 3 points
Removed measurement 2c285648 to history. Total: 3
th(0)=-2.3820961150364908;dx=-7.250079644956541E-5
Armijo: th(54291.75418880348)=-1.6390462861653767; dx=1.0582160183804878E-5 evalInputDelta=-0.743049828871114
New Minimum: -2.3820961150364908 > -2.650947257155352
WOLF (strong): th(27145.87709440174)=-2.650947257155352; dx=1.2968953993259656E-6 evalInputDelta=0.2688511421188613
New Minimum: -2.650947257155352 > -2.843802955904097
END: th(9048.625698133914)=-2.843802955904097; dx=-3.1139282946834854E-5 evalInputDelta=0.46170684086760616
Fitness changed from -2.3820961150364908 to -2.843802955904097
Iteration 4 complete. Error: -2.843802955904097 Total: 0.6107; Orientation: 0.0273; Line Search: 0.4627
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 5731a4a5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.905e+03, gradient 6.311e-03, dot -0.307; [1b8511b7-b137-4dac-b07f-4e496baa8372 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.843802955904097, -1.7985733481713146, -0.9082399929040347, -0.3807669467139844
LBFGS Accumulation History: 3 points
Removed measurement 5731a4a5 to history. Total: 3
th(0)=-2.843802955904097;dx=-3.898351178127577E-5
New Minimum: -2.843802955904097 > -3.1797973368656542
END: th(19494.673101173677)=-3.1797973368656542; dx=-1.5554823786163057E-5 evalInputDelta=0.3359943809615573
Fitness changed from -2.843802955904097 to -3.1797973368656542
Iteration 5 complete. Error: -3.1797973368656542 Total: 0.3737; Orientation: 0.0289; Line Search: 0.2242
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 1139c8be to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.824e+03, gradient 6.525e-03, dot -0.324; [1b8511b7-b137-4dac-b07f-4e496baa8372 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.1797973368656542, -1.7985733481713146, -0.9082399929040347, -0.3807669467139844
LBFGS Accumulation History: 3 points
Removed measurement 1139c8be to history. Total: 3
th(0)=-3.1797973368656542;dx=-4.078519428931818E-5
New Minimum: -3.1797973368656542 > -3.3643110845702533
WOLF (strong): th(42000.000000000015)=-3.3643110845702533; dx=1.5233965883938483E-6 evalInputDelta=0.18451374770459905
New Minimum: -3.3643110845702533 > -3.5888352182338985
END: th(21000.000000000007)=-3.5888352182338985; dx=-9.193942554701835E-6 evalInputDelta=0.40903788136824426
Fitness changed from -3.1797973368656542 to -3.5888352182338985
Iteration 6 complete. Error: -3.5888352182338985 Total: 0.4904; Orientation: 0.0276; Line Search: 0.3424
Adding measurement 5ca5e214 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.389e+03, gradient 7.084e-03, dot -0.243; [1b8511b7-b137-4dac-b07f-4e496baa8372 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.5888352182338985, -1.7985733481713146, -0.9082399929040347, -0.3807669467139844
LBFGS Accumulation History: 3 points
Removed measurement 5ca5e214 to history. Total: 3
th(0)=-3.5888352182338985;dx=-4.829311949952782E-5
New Minimum: -3.5888352182338985 > -3.7171566580872737
END: th(45243.12849066957)=-3.7171566580872737; dx=-4.584455477047598E-7 evalInputDelta=0.12832143985337519
Fitness changed from -3.5888352182338985 to -3.7171566580872737
Iteration 7 complete. Error: -3.7171566580872737 Total: 0.3602; Orientation: 0.0267; Line Search: 0.2230
Adding measurement 4455733b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.803e+03, gradient 8.158e-03, dot -0.235; [1b8511b7-b137-4dac-b07f-4e496baa8372 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.7171566580872737, -1.7985733481713146, -0.9082399929040347, -0.3807669467139844
LBFGS Accumulation History: 3 points
Removed measurement 4455733b to history. Total: 3
th(0)=-3.7171566580872737;dx=-6.456128828725554E-5
Armijo: th(97473.3655058684)=-2.8440527033777605; dx=1.049796134324842E-5 evalInputDelta=-0.8731039547095132
New Minimum: -3.7171566580872737 > -4.32148298391093
WOLF (strong): th(48736.6827529342)=-4.32148298391093; dx=8.6560015516837E-6 evalInputDelta=0.6043263258236564
New Minimum: -4.32148298391093 > -4.438393127007272
END: th(16245.560917644732)=-4.438393127007272; dx=-2.1163135791891668E-5 evalInputDelta=0.7212364689199982
Fitness changed from -3.7171566580872737 to -4.438393127007272
Iteration 8 complete. Error: -4.438393127007272 Total: 0.5995; Orientation: 0.0292; Line Search: 0.4567
Adding measurement 426ec750 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.355e+03, gradient 6.281e-03, dot -0.231; [1b8511b7-b137-4dac-b07f-4e496baa8372 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.438393127007272, -1.7985733481713146, -0.9082399929040347, -0.3807669467139844
LBFGS Accumulation History: 3 points
Removed measurement 426ec750 to history. Total: 3
th(0)=-4.438393127007272;dx=-3.7673997592503217E-5
New Minimum: -4.438393127007272 > -5.002892158578531
END: th(35000.000000000015)=-5.002892158578531; dx=-1.5649421412731558E-6 evalInputDelta=0.564499031571259
Fitness changed from -4.438393127007272 to -5.002892158578531
Iteration 9 complete. Error: -5.002892158578531 Total: 0.3696; Orientation: 0.0280; Line Search: 0.2267
Adding measurement 9060eb0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.908e+03, gradient 7.573e-03, dot -0.262; [1b8511b7-b137-4dac-b07f-4e496baa8372 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.002892158578531, -1.7985733481713146, -0.9082399929040347, -0.3807669467139844
LBFGS Accumulation History: 3 points
Removed measurement 9060eb0 to history. Total: 3
th(0)=-5.002892158578531;dx=-5.6559795718365003E-5
Armijo: th(75405.21415111597)=-4.374682228247364; dx=1.2677857700392328E-5 evalInputDelta=-0.6282099303311668
New Minimum: -5.002892158578531 > -5.192335582395387
WOLF (strong): th(37702.607075557986)=-5.192335582395387; dx=6.99397250804069E-6 evalInputDelta=0.18944342381685608
New Minimum: -5.192335582395387 > -5.446100345950362
END: th(12567.535691852661)=-5.446100345950362; dx=-2.022364811376055E-5 evalInputDelta=0.4432081873718312
Fitness changed from -5.002892158578531 to -5.446100345950362
Iteration 10 complete. Error: -5.446100345950362 Total: 0.5855; Orientation: 0.0283; Line Search: 0.4461
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 753a939c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.859e+03, gradient 5.224e-03, dot -0.336; [1b8511b7-b137-4dac-b07f-4e496baa8372 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.446100345950362, -1.7985733481713146, -0.9082399929040347, -0.3807669467139844
LBFGS Accumulation History: 3 points
Removed measurement 753a939c to history. Total: 3
th(0)=-5.446100345950362;dx=-2.5521480006573265E-5
New Minimum: -5.446100345950362 > -5.8781524889869115
END: th(27075.934862741226)=-5.8781524889869115; dx=-1.593457835145259E-5 evalInputDelta=0.4320521430365494
Fitness changed from -5.446100345950362 to -5.8781524889869115
Iteration 11 complete. Error: -5.8781524889869115 Total: 0.3792; Orientation: 0.0278; Line Search: 0.2267
Adding measurement 3830184e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.186e+03, gradient 5.240e-03, dot -0.315; [1b8511b7-b137-4dac-b07f-4e496baa8372 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.8781524889869115, -1.7985733481713146, -0.9082399929040347, -0.3807669467139844
LBFGS Accumulation History: 3 points
Removed measurement 3830184e to history. Total: 3
th(0)=-5.8781524889869115;dx=-2.6236568546865407E-5
New Minimum: -5.8781524889869115 > -5.991452489985049
END: th(58333.33333333337)=-5.991452489985049; dx=-6.841298229255351E-6 evalInputDelta=0.1133000009981373
Fitness changed from -5.8781524889869115 to -5.991452489985049
Iteration 12 complete. Error: -5.991452489985049 Total: 0.3611; Orientation: 0.0287; Line Search: 0.2263
Adding measurement 4ce2179f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.676e+03, gradient 7.222e-03, dot -0.281; [1b8511b7-b137-4dac-b07f-4e496baa8372 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.991452489985049, -1.7985733481713146, -0.9082399929040347, -0.3807669467139844
LBFGS Accumulation History: 3 points
Removed measurement 4ce2179f to history. Total: 3
th(0)=-5.991452489985049;dx=-5.055461786340611E-5
Armijo: th(125675.35691852664)=-3.9674549545474758; dx=5.917706706959442E-6 evalInputDelta=-2.023997535437573
Armijo: th(62837.67845926332)=-5.665424138204699; dx=1.0171001549659809E-5 evalInputDelta=-0.3260283517803497
New Minimum: -5.991452489985049 > -6.626044008574805
END: th(20945.89281975444)=-6.626044008574805; dx=-1.588787916696902E-5 evalInputDelta=0.6345915185897564
Fitness changed from -5.991452489985049 to -6.626044008574805
Iteration 13 complete. Error: -6.626044008574805 Total: 0.5970; Orientation: 0.0275; Line Search: 0.4582
Adding measurement 46de4e20 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.408e+03, gradient 5.621e-03, dot -0.325; [1b8511b7-b137-4dac-b07f-4e496baa8372 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.626044008574805, -1.7985733481713146, -0.9082399929040347, -0.3807669467139844
LBFGS Accumulation History: 3 points
Removed measurement 46de4e20 to history. Total: 3
th(0)=-6.626044008574805;dx=-2.9866219437172035E-5
New Minimum: -6.626044008574805 > -7.115065586915862
END: th(45126.55810456872)=-7.115065586915862; dx=-8.11417651947974E-6 evalInputDelta=0.489021578341057
Fitness changed from -6.626044008574805 to -7.115065586915862
Iteration 14 complete. Error: -7.115065586915862 Total: 0.3607; Orientation: 0.0286; Line Search: 0.2249
Adding measurement 6ebe001d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.469e+04, gradient 6.993e-03, dot -0.304; [1b8511b7-b137-4dac-b07f-4e496baa8372 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -7.115065586915862, -1.7985733481713146, -0.9082399929040347, -0.3807669467139844
LBFGS Accumulation History: 3 points
Removed measurement 6ebe001d to history. Total: 3
th(0)=-7.115065586915862;dx=-4.704077008302469E-5
Armijo: th(97222.2222222223)=-5.460593486005229; dx=3.0878341159941326E-6 evalInputDelta=-1.654472100910633
Armijo: th(48611.11111111115)=-7.01582396587252; dx=5.454424063561155E-6 evalInputDelta=-0.09924162104334222
New Minimum: -7.115065586915862 > -7.565364231638445
END: th(16203.703703703717)=-7.565364231638445; dx=-1.8928421344918393E-5 evalInputDelta=0.45029864472258296
Fitness changed from -7.115065586915862 to -7.565364231638445
Iteration 15 complete. Error: -7.565364231638445 Total: 0.5909; Orientation: 0.0273; Line Search: 0.4521
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -7.565364231638445 (> -Infinity) after 7.958s (< 5400.000s)
Returns
-7.565364231638445