BasicOptimizer.scala:89 executed in 71.13 seconds (1.399 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 154994327945700
Reset training subject: 154995929642600
Adding measurement 7bf8e282 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-121.67862384165922;dx=-0.018832917622244785
New Minimum: -121.67862384165922 > -121.7177992581611
WOLFE (weak): th(2.154434690031884)=-121.7177992581611; dx=-0.01842293156950655 evalInputDelta=0.039175416501876725
New Minimum: -121.7177992581611 > -121.75698613170006
WOLFE (weak): th(4.308869380063768)=-121.75698613170006; dx=-0.01837227214551681 evalInputDelta=0.07836229004084316
New Minimum: -121.75698613170006 > -121.91374384170146
WOLFE (weak): th(12.926608140191302)=-121.91374384170146; dx=-0.01839593480294795 evalInputDelta=0.23512000004224376
New Minimum: -121.91374384170146 > -122.61963625501627
WOLFE (weak): th(51.70643256076521)=-122.61963625501627; dx=-0.018441923110086064 evalInputDelta=0.9410124133570577
New Minimum: -122.61963625501627 > -126.41452769559359
WOLFE (weak): th(258.53216280382605)=-126.41452769559359; dx=-0.018613280492998107 evalInputDelta=4.735903853934374
New Minimum: -126.41452769559359 > -150.87654695768657
WOLFE (weak): th(1551.1929768229563)=-150.87654695768657; dx=-0.018353923747900038 evalInputDelta=29.197923116027354
New Minimum: -150.87654695768657 > -289.7758023455375
END: th(10858.350837760694)=-289.7758023455375; dx=-0.010400254969017537 evalInputDelta=168.0971785038783
Fitness changed from -121.67862384165922 to -289.7758023455375
Iteration 1 complete. Error: -289.7758023455375 Total: 10.4708; Orientation: 0.0215; Line Search: 7.0426
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement dc3888c to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-289.7758023455375;dx=-0.021706740925658483
New Minimum: -289.7758023455375 > -589.6666696108913
END: th(23393.607721408407)=-589.6666696108913; dx=-0.0065170875351991744 evalInputDelta=299.8908672653538
Fitness changed from -289.7758023455375 to -589.6666696108913
Iteration 2 complete. Error: -589.6666696108913 Total: 2.8122; Orientation: 0.0144; Line Search: 1.7467
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 77967ab8 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-589.6666696108913;dx=-0.024234230293329984
New Minimum: -589.6666696108913 > -901.8593986869572
END: th(50400.00000000001)=-901.8593986869572; dx=-0.0021497660470189274 evalInputDelta=312.1927290760659
Fitness changed from -589.6666696108913 to -901.8593986869572
Iteration 3 complete. Error: -901.8593986869572 Total: 2.5881; Orientation: 0.0148; Line Search: 1.7006
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 7ef42e1a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.058e+04, gradient 1.711e-01, dot -0.211; [708f7273-f7d5-4c45-9078-64d3cb29ab8b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -901.8593986869572, -589.6666696108913, -289.7758023455375, -121.67862384165922
LBFGS Accumulation History: 3 points
Removed measurement 7ef42e1a to history. Total: 3
th(0)=-901.8593986869572;dx=-0.02909583591593852
New Minimum: -901.8593986869572 > -998.7868891810705
WOLF (strong): th(108583.50837760697)=-998.7868891810705; dx=8.719099442246615E-4 evalInputDelta=96.92749049411327
New Minimum: -998.7868891810705 > -1273.6444755111215
END: th(54291.75418880348)=-1273.6444755111215; dx=-0.001258002863038859 evalInputDelta=371.7850768241643
Fitness changed from -901.8593986869572 to -1273.6444755111215
Iteration 4 complete. Error: -1273.6444755111215 Total: 3.7208; Orientation: 0.2801; Line Search: 2.5222
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 5ad7a6d6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.561e+04, gradient 1.447e-01, dot -0.178; [708f7273-f7d5-4c45-9078-64d3cb29ab8b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1273.6444755111215, -589.6666696108913, -289.7758023455375, -121.67862384165922
LBFGS Accumulation History: 3 points
Removed measurement 5ad7a6d6 to history. Total: 3
th(0)=-1273.6444755111215;dx=-0.020884691599531338
New Minimum: -1273.6444755111215 > -1442.9387547640406
WOLF (strong): th(116968.03860704206)=-1442.9387547640406; dx=0.0010375967524937902 evalInputDelta=169.29427925291907
New Minimum: -1442.9387547640406 > -1701.740065344881
END: th(58484.01930352103)=-1701.740065344881; dx=-0.0030269125461890338 evalInputDelta=428.0955898337595
Fitness changed from -1273.6444755111215 to -1701.740065344881
Iteration 5 complete. Error: -1701.740065344881 Total: 3.8774; Orientation: 0.2740; Line Search: 2.5105
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 79f00dd6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.263e+04, gradient 1.358e-01, dot -0.190; [708f7273-f7d5-4c45-9078-64d3cb29ab8b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1701.740065344881, -589.6666696108913, -289.7758023455375, -121.67862384165922
LBFGS Accumulation History: 3 points
Removed measurement 79f00dd6 to history. Total: 3
th(0)=-1701.740065344881;dx=-0.0183795263234508
New Minimum: -1701.740065344881 > -1781.0961512506842
WOLF (strong): th(126000.00000000004)=-1781.0961512506842; dx=9.545382522678563E-4 evalInputDelta=79.35608590580318
New Minimum: -1781.0961512506842 > -2056.291564929448
END: th(63000.00000000002)=-2056.291564929448; dx=-0.0030906575790115316 evalInputDelta=354.5514995845672
Fitness changed from -1701.740065344881 to -2056.291564929448
Iteration 6 complete. Error: -2056.291564929448 Total: 4.0891; Orientation: 0.3281; Line Search: 2.7744
Adding measurement 45cc7496 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.898e+04, gradient 1.342e-01, dot -0.214; [708f7273-f7d5-4c45-9078-64d3cb29ab8b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2056.291564929448, -589.6666696108913, -289.7758023455375, -121.67862384165922
LBFGS Accumulation History: 3 points
Removed measurement 45cc7496 to history. Total: 3
th(0)=-2056.291564929448;dx=-0.017942543593551477
Armijo: th(135729.38547200873)=-1981.605725163308; dx=0.0011389069686283484 evalInputDelta=-74.68583976614013
New Minimum: -2056.291564929448 > -2349.937765133366
END: th(67864.69273600436)=-2349.937765133366; dx=-0.0021785288658677936 evalInputDelta=293.64620020391794
Fitness changed from -2056.291564929448 to -2349.937765133366
Iteration 7 complete. Error: -2349.937765133366 Total: 3.9793; Orientation: 0.3033; Line Search: 2.8168
Adding measurement 4f3a8819 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.976e+04, gradient 1.404e-01, dot -0.240; [708f7273-f7d5-4c45-9078-64d3cb29ab8b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2349.937765133366, -589.6666696108913, -289.7758023455375, -121.67862384165922
LBFGS Accumulation History: 3 points
Removed measurement 4f3a8819 to history. Total: 3
th(0)=-2349.937765133366;dx=-0.019674532589502414
Armijo: th(146210.0482588026)=-2015.0281635748916; dx=0.0021267912761238914 evalInputDelta=-334.9096015584746
New Minimum: -2349.937765133366 > -2557.5971144806995
END: th(73105.0241294013)=-2557.5971144806995; dx=-4.96282597687692E-4 evalInputDelta=207.65934934733332
Fitness changed from -2349.937765133366 to -2557.5971144806995
Iteration 8 complete. Error: -2557.5971144806995 Total: 3.8736; Orientation: 0.2812; Line Search: 2.6461
Adding measurement 59b21c4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.359e+05, gradient 1.467e-01, dot -0.232; [708f7273-f7d5-4c45-9078-64d3cb29ab8b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2557.5971144806995, -589.6666696108913, -289.7758023455375, -121.67862384165922
LBFGS Accumulation History: 3 points
Removed measurement 59b21c4 to history. Total: 3
th(0)=-2557.5971144806995;dx=-0.021429336151897044
Armijo: th(157500.00000000006)=-2093.3397956375834; dx=0.003379051710112375 evalInputDelta=-464.2573188431161
New Minimum: -2557.5971144806995 > -2777.6107538674314
WOLF (strong): th(78750.00000000003)=-2777.6107538674314; dx=0.0011526317190810802 evalInputDelta=220.01363938673194
New Minimum: -2777.6107538674314 > -2866.912189098229
END: th(26250.00000000001)=-2866.912189098229; dx=-0.007291560187712971 evalInputDelta=309.3150746175297
Fitness changed from -2557.5971144806995 to -2866.912189098229
Iteration 9 complete. Error: -2866.912189098229 Total: 4.6190; Orientation: 0.2800; Line Search: 3.5175
Adding measurement 1bb794f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.144e+05, gradient 1.173e-01, dot -0.311; [708f7273-f7d5-4c45-9078-64d3cb29ab8b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2866.912189098229, -589.6666696108913, -289.7758023455375, -121.67862384165922
LBFGS Accumulation History: 3 points
Removed measurement 1bb794f to history. Total: 3
th(0)=-2866.912189098229;dx=-0.013688475235171593
New Minimum: -2866.912189098229 > -3069.656658325162
END: th(56553.91061333697)=-3069.656658325162; dx=-0.004840807078607694 evalInputDelta=202.74446922693278
Fitness changed from -2866.912189098229 to -3069.656658325162
Iteration 10 complete. Error: -3069.656658325162 Total: 3.0837; Orientation: 0.4246; Line Search: 1.8539
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 1120f89b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.989e+06, gradient 1.223e-01, dot -0.327; [708f7273-f7d5-4c45-9078-64d3cb29ab8b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3069.656658325162, -589.6666696108913, -289.7758023455375, -121.67862384165922
LBFGS Accumulation History: 3 points
Removed measurement 1120f89b to history. Total: 3
th(0)=-3069.656658325162;dx=-0.014818197368689695
New Minimum: -3069.656658325162 > -3154.24967306014
END: th(121841.70688233551)=-3154.24967306014; dx=-8.736495626345893E-4 evalInputDelta=84.59301473497817
Fitness changed from -3069.656658325162 to -3154.24967306014
Iteration 11 complete. Error: -3154.24967306014 Total: 3.0889; Orientation: 0.2919; Line Search: 1.8728
Adding measurement 4e40fddf to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.115e+07, gradient 1.448e-01, dot -0.237; [708f7273-f7d5-4c45-9078-64d3cb29ab8b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3154.24967306014, -589.6666696108913, -289.7758023455375, -121.67862384165922
LBFGS Accumulation History: 3 points
Removed measurement 4e40fddf to history. Total: 3
th(0)=-3154.24967306014;dx=-0.020924451801377968
Armijo: th(262500.0000000001)=-1607.4228597354213; dx=0.002820364923730727 evalInputDelta=-1546.8268133247188
Armijo: th(131250.00000000006)=-2878.0549855761965; dx=0.004103346514733762 evalInputDelta=-276.19468748394365
New Minimum: -3154.24967306014 > -3518.12052096002
END: th(43750.00000000002)=-3518.12052096002; dx=-0.004746456179980484 evalInputDelta=363.87084789987966
Fitness changed from -3154.24967306014 to -3518.12052096002
Iteration 12 complete. Error: -3518.12052096002 Total: 4.6166; Orientation: 0.3119; Line Search: 3.4966
Adding measurement 7c546c5b to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.007e+05, gradient 1.179e-01, dot 0.388; [708f7273-f7d5-4c45-9078-64d3cb29ab8b = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-3518.12052096002;dx=-4607.393673792671
Armijo: th(94256.51768889496)=-1248.880849000416; dx=0.008651631318353893 evalInputDelta=-2269.2396719596036
Armijo: th(47128.25884444748)=-1248.889356588485; dx=0.017343617288460404 evalInputDelta=-2269.2311643715348
Armijo: th(15709.419614815828)=-1248.8053791524121; dx=0.05200016309626844 evalInputDelta=-2269.3151418076077
Armijo: th(3927.354903703957)=-1248.674647376169; dx=0.20753672420124325 evalInputDelta=-2269.4458735838507
Armijo: th(785.4709807407914)=-1249.584030787846; dx=1.0375677936316063 evalInputDelta=-2268.536490172174
Armijo: th(130.91183012346525)=-1262.6536045170901; dx=6.260815504560098 evalInputDelta=-2255.4669164429297
Armijo: th(18.701690017637894)=-1339.335236227286; dx=44.682732231350485 evalInputDelta=-2178.785284732734
Armijo: th(2.3377112522047367)=-1831.4668301664653; dx=387.11449588303657 evalInputDelta=-1686.6536907935545
Armijo: th(0.2597456946894152)=-3365.536509978758; dx=771.3756065400764 evalInputDelta=-152.5840109812616
New Minimum: -3518.12052096002 > -3562.196513388895
END: th(0.02597456946894152)=-3562.196513388895; dx=-1593.8536479048005 evalInputDelta=44.075992428875
Fitness changed from -3518.12052096002 to -3562.196513388895
Iteration 13 complete. Error: -3562.196513388895 Total: 10.6454; Orientation: 0.3971; Line Search: 9.4661
Adding measurement 7b4b625f to history. Total: 4
Accepted: LBFGS Orientation magnitude: 4.287e+04, gradient 1.184e-01, dot 0.372; [708f7273-f7d5-4c45-9078-64d3cb29ab8b = 1.000/1.000e+00]
th(0)=-3562.196513388895;dx=-1890.1700281175385
New Minimum: -3562.196513388895 > -3569.453739719847
END: th(0.05596051352253066)=-3569.453739719847; dx=-234.20843556687828 evalInputDelta=7.257226330952108
Fitness changed from -3562.196513388895 to -3569.453739719847
Iteration 14 complete. Error: -3569.453739719847 Total: 3.1431; Orientation: 0.3401; Line Search: 1.8551
Adding measurement 2b8a9763 to history. Total: 5
Accepted: LBFGS Orientation magnitude: 3.907e+04, gradient 1.202e-01, dot 0.339; [708f7273-f7d5-4c45-9078-64d3cb29ab8b = 1.000/1.000e+00]
th(0)=-3569.453739719847;dx=-1590.5411740170402
Armijo: th(0.12056327160493839)=-3550.8881987468994; dx=-5.988179530794444 evalInputDelta=-18.565540972947474
Armijo: th(0.060281635802469195)=-3564.6255062769214; dx=-92.63468158820444 evalInputDelta=-4.828233442925466
Armijo: th(0.020093878600823064)=-3568.903192859117; dx=-152.98842932011172 evalInputDelta=-0.5505468607298099
Armijo: th(0.005023469650205766)=-3569.436650229277; dx=-180.81093201180053 evalInputDelta=-0.017089490570015187
New Minimum: -3569.453739719847 > -3569.456401190523
END: th(0.0010046939300411531)=-3569.456401190523; dx=-188.36642163761223 evalInputDelta=0.0026614706762302376
Fitness changed from -3569.453739719847 to -3569.456401190523
Iteration 15 complete. Error: -3569.456401190523 Total: 6.4114; Orientation: 0.4181; Line Search: 5.1771
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -3569.456401190523 (> -Infinity) after 71.132s (< 5400.000s)
Returns
-3569.456401190523