BasicOptimizer.scala:89 executed in 47.23 seconds (0.801 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 217778929903500
Reset training subject: 217779565074700
Adding measurement 2500e5ab to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-210.53745398681443;dx=-0.06055406269404066
New Minimum: -210.53745398681443 > -210.64331322028156
WOLFE (weak): th(2.154434690031884)=-210.64331322028156; dx=-0.060881014839665924 evalInputDelta=0.10585923346712889
New Minimum: -210.64331322028156 > -210.74930432579833
WOLFE (weak): th(4.308869380063768)=-210.74930432579833; dx=-0.060772310954371786 evalInputDelta=0.2118503389839077
New Minimum: -210.74930432579833 > -211.17293718204454
WOLFE (weak): th(12.926608140191302)=-211.17293718204454; dx=-0.060681089600569275 evalInputDelta=0.6354831952301083
New Minimum: -211.17293718204454 > -213.07354996014203
WOLFE (weak): th(51.70643256076521)=-213.07354996014203; dx=-0.06016903645145559 evalInputDelta=2.5360959733276047
New Minimum: -213.07354996014203 > -223.34531754606283
WOLFE (weak): th(258.53216280382605)=-223.34531754606283; dx=-0.05929605955192055 evalInputDelta=12.807863559248403
New Minimum: -223.34531754606283 > -284.50626848100313
END: th(1551.1929768229563)=-284.50626848100313; dx=-0.04710257612952053 evalInputDelta=73.9688144941887
Fitness changed from -210.53745398681443 to -284.50626848100313
Iteration 1 complete. Error: -284.50626848100313 Total: 6.0173; Orientation: 0.0159; Line Search: 4.2491
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 18d75ab2 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-284.50626848100313;dx=-0.061619323751532715
New Minimum: -284.50626848100313 > -404.4499688038809
END: th(3341.943960201201)=-404.4499688038809; dx=-0.030177100720058005 evalInputDelta=119.94370032287776
Fitness changed from -284.50626848100313 to -404.4499688038809
Iteration 2 complete. Error: -404.4499688038809 Total: 2.1453; Orientation: 0.0169; Line Search: 1.3746
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 1fb9f8ad to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-404.4499688038809;dx=-0.07005892234110708
New Minimum: -404.4499688038809 > -562.7085163659856
END: th(7200.000000000001)=-562.7085163659856; dx=-0.017541028098379242 evalInputDelta=158.2585475621047
Fitness changed from -404.4499688038809 to -562.7085163659856
Iteration 3 complete. Error: -562.7085163659856 Total: 1.9469; Orientation: 0.0165; Line Search: 1.3333
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 272a7163 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.029e+04, gradient 2.777e-01, dot -0.260; [8b318c05-5131-4be9-a2e8-50d616ccd140 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -562.7085163659856, -404.4499688038809, -284.50626848100313, -210.53745398681443
LBFGS Accumulation History: 3 points
Removed measurement 272a7163 to history. Total: 3
th(0)=-562.7085163659856;dx=-0.07665012113734257
New Minimum: -562.7085163659856 > -691.7413725830165
END: th(15511.929768229566)=-691.7413725830165; dx=-0.008115579369456278 evalInputDelta=129.03285621703094
Fitness changed from -562.7085163659856 to -691.7413725830165
Iteration 4 complete. Error: -691.7413725830165 Total: 2.2350; Orientation: 0.3069; Line Search: 1.3052
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 603d352a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.447e+03, gradient 3.063e-01, dot -0.166; [8b318c05-5131-4be9-a2e8-50d616ccd140 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -691.7413725830165, -404.4499688038809, -284.50626848100313, -210.53745398681443
LBFGS Accumulation History: 3 points
Removed measurement 603d352a to history. Total: 3
th(0)=-691.7413725830165;dx=-0.09373620370295914
New Minimum: -691.7413725830165 > -735.6358517780909
END: th(33419.43960201202)=-735.6358517780909; dx=-0.0018911359111063574 evalInputDelta=43.89447919507438
Fitness changed from -691.7413725830165 to -735.6358517780909
Iteration 5 complete. Error: -735.6358517780909 Total: 2.1478; Orientation: 0.3022; Line Search: 1.2366
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 3bcebc06 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.650e+03, gradient 2.819e-01, dot -0.117; [8b318c05-5131-4be9-a2e8-50d616ccd140 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -735.6358517780909, -404.4499688038809, -284.50626848100313, -210.53745398681443
LBFGS Accumulation History: 3 points
Removed measurement 3bcebc06 to history. Total: 3
th(0)=-735.6358517780909;dx=-0.07937522024424644
Armijo: th(72000.00000000003)=-677.6896854849915; dx=7.213790643309872E-5 evalInputDelta=-57.946166293099395
New Minimum: -735.6358517780909 > -1075.10417807251
END: th(36000.000000000015)=-1075.10417807251; dx=-0.005827980426837314 evalInputDelta=339.46832629441906
Fitness changed from -735.6358517780909 to -1075.10417807251
Iteration 6 complete. Error: -1075.10417807251 Total: 2.7184; Orientation: 0.2928; Line Search: 1.8399
Adding measurement 4d78b66d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.071e+04, gradient 2.653e-01, dot -0.132; [8b318c05-5131-4be9-a2e8-50d616ccd140 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1075.10417807251, -404.4499688038809, -284.50626848100313, -210.53745398681443
LBFGS Accumulation History: 3 points
Removed measurement 4d78b66d to history. Total: 3
th(0)=-1075.10417807251;dx=-0.07030560313962159
Armijo: th(77559.64884114785)=-946.6429588328408; dx=6.471265560367978E-4 evalInputDelta=-128.46121923966916
New Minimum: -1075.10417807251 > -1469.0800737049374
END: th(38779.824420573925)=-1469.0800737049374; dx=-0.008517101002216889 evalInputDelta=393.9758956324274
Fitness changed from -1075.10417807251 to -1469.0800737049374
Iteration 7 complete. Error: -1469.0800737049374 Total: 3.0584; Orientation: 0.3126; Line Search: 2.1496
Adding measurement 3a6b9471 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.970e+04, gradient 2.663e-01, dot -0.156; [8b318c05-5131-4be9-a2e8-50d616ccd140 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1469.0800737049374, -404.4499688038809, -284.50626848100313, -210.53745398681443
LBFGS Accumulation History: 3 points
Removed measurement 3a6b9471 to history. Total: 3
th(0)=-1469.0800737049374;dx=-0.07076976642883254
Armijo: th(83548.59900503006)=-1201.9324760927227; dx=0.0011475085696080467 evalInputDelta=-267.14759761221467
New Minimum: -1469.0800737049374 > -1825.258596256605
END: th(41774.29950251503)=-1825.258596256605; dx=-0.008623893498567535 evalInputDelta=356.1785225516676
Fitness changed from -1469.0800737049374 to -1825.258596256605
Iteration 8 complete. Error: -1825.258596256605 Total: 2.6639; Orientation: 0.2813; Line Search: 1.8239
Adding measurement 3f747c69 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.473e+05, gradient 2.715e-01, dot -0.173; [8b318c05-5131-4be9-a2e8-50d616ccd140 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1825.258596256605, -404.4499688038809, -284.50626848100313, -210.53745398681443
LBFGS Accumulation History: 3 points
Removed measurement 3f747c69 to history. Total: 3
th(0)=-1825.258596256605;dx=-0.07359262204851738
Armijo: th(90000.00000000004)=-1361.7810474313037; dx=0.002025644695221463 evalInputDelta=-463.47754882530126
New Minimum: -1825.258596256605 > -2127.844196613667
END: th(45000.00000000002)=-2127.844196613667; dx=-0.007215278470595301 evalInputDelta=302.58560035706205
Fitness changed from -1825.258596256605 to -2127.844196613667
Iteration 9 complete. Error: -2127.844196613667 Total: 2.6299; Orientation: 0.2899; Line Search: 1.7667
Adding measurement 16fe300d to history. Total: 3
Accepted: LBFGS Orientation magnitude: 8.157e+04, gradient 2.823e-01, dot 0.196; [8b318c05-5131-4be9-a2e8-50d616ccd140 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-2127.844196613667;dx=-4523.619799551945
Armijo: th(96949.56105143482)=-64.78386453735845; dx=0.011423994564878037 evalInputDelta=-2063.0603320763084
Armijo: th(48474.78052571741)=-64.78109827117237; dx=0.02283466306646018 evalInputDelta=-2063.0630983424944
Armijo: th(16158.260175239137)=-64.7733672752671; dx=0.06869258173309722 evalInputDelta=-2063.0708293384
Armijo: th(4039.565043809784)=-64.90584297981309; dx=0.2739749152207321 evalInputDelta=-2062.9383536338537
Armijo: th(807.9130087619568)=-66.11729938054032; dx=1.3725378528243617 evalInputDelta=-2061.7268972331267
Armijo: th(134.6521681269928)=-71.27185976865212; dx=8.113001009330189 evalInputDelta=-2056.572336845015
Armijo: th(19.236024018141826)=-124.81284838340105; dx=52.690480307521405 evalInputDelta=-2003.0313482302658
Armijo: th(2.4045030022677283)=-499.6098842638046; dx=270.3445863834643 evalInputDelta=-1628.2343123498624
Armijo: th(0.2671670002519698)=-1776.6918025107577; dx=-13.366520864994527 evalInputDelta=-351.1523941029093
Armijo: th(0.026716700025196983)=-2126.9268543310127; dx=-1558.3170798556366 evalInputDelta=-0.9173422826543174
New Minimum: -2127.844196613667 > -2128.5051163520484
END: th(0.0024287909113815437)=-2128.5051163520484; dx=-1951.1887374283706 evalInputDelta=0.6609197383813807
Fitness changed from -2127.844196613667 to -2128.5051163520484
Iteration 10 complete. Error: -2128.5051163520484 Total: 8.3393; Orientation: 0.2866; Line Search: 7.5115
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 4d8483b4 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 6.136e+06, gradient 2.819e-01, dot -0.169; [8b318c05-5131-4be9-a2e8-50d616ccd140 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2128.5051163520484, -2127.844196613667, -404.4499688038809, -284.50626848100313, -210.53745398681443
Rejected: LBFGS Orientation magnitude: 1.093e+07, gradient 2.819e-01, dot -0.174; [8b318c05-5131-4be9-a2e8-50d616ccd140 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2128.5051163520484, -2127.844196613667, -404.4499688038809, -284.50626848100313
LBFGS Accumulation History: 3 points
Removed measurement 4d8483b4 to history. Total: 4
Removed measurement 16fe300d to history. Total: 3
th(0)=-2128.5051163520484;dx=-0.07931273830986883
New Minimum: -2128.5051163520484 > -2128.5053412633706
END: th(0.005232671394314553)=-2128.5053412633706; dx=-0.06453099725947262 evalInputDelta=2.2491132222057786E-4
Fitness changed from -2128.5051163520484 to -2128.5053412633706
Iteration 11 complete. Error: -2128.5053412633706 Total: 2.3990; Orientation: 0.6293; Line Search: 1.1999
Adding measurement 38e383cc to history. Total: 3
Accepted: LBFGS Orientation magnitude: 8.194e+04, gradient 2.818e-01, dot 0.196; [8b318c05-5131-4be9-a2e8-50d616ccd140 = 1.000/1.000e+00]
th(0)=-2128.5053412633706;dx=-4532.217047235883
New Minimum: -2128.5053412633706 > -2129.520053693271
END: th(0.011273448773448779)=-2129.520053693271; dx=-1768.969920926807 evalInputDelta=1.0147124299005554
Fitness changed from -2128.5053412633706 to -2129.520053693271
Iteration 12 complete. Error: -2129.520053693271 Total: 1.9875; Orientation: 0.2914; Line Search: 1.1593
Adding measurement 6c308bda to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.188e+06, gradient 2.794e-01, dot -0.164; [8b318c05-5131-4be9-a2e8-50d616ccd140 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2129.520053693271, -2128.5053412633706, -404.4499688038809, -284.50626848100313, -210.53745398681443
Rejected: LBFGS Orientation magnitude: 1.832e+06, gradient 2.794e-01, dot -0.171; [8b318c05-5131-4be9-a2e8-50d616ccd140 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2129.520053693271, -2128.5053412633706, -404.4499688038809, -284.50626848100313
LBFGS Accumulation History: 3 points
Removed measurement 6c308bda to history. Total: 4
Removed measurement 38e383cc to history. Total: 3
th(0)=-2129.520053693271;dx=-0.07790863679638217
New Minimum: -2129.520053693271 > -2129.521063305694
END: th(0.02428790911381544)=-2129.521063305694; dx=-0.06295517338986376 evalInputDelta=0.0010096124228766712
Fitness changed from -2129.520053693271 to -2129.521063305694
Iteration 13 complete. Error: -2129.521063305694 Total: 2.6184; Orientation: 0.6412; Line Search: 1.4194
Adding measurement 1389cb8e to history. Total: 3
Accepted: LBFGS Orientation magnitude: 7.219e+04, gradient 2.795e-01, dot 0.201; [8b318c05-5131-4be9-a2e8-50d616ccd140 = 1.000/1.000e+00]
th(0)=-2129.521063305694;dx=-4042.2302713227054
Armijo: th(0.05232671394314553)=-2109.8528462292347; dx=-1183.6311637078986 evalInputDelta=-19.668217076459314
Armijo: th(0.026163356971572765)=-2124.762404399008; dx=-1435.508200903928 evalInputDelta=-4.758658906685923
Armijo: th(0.008721118990524255)=-2129.131383396838; dx=-1605.757710530791 evalInputDelta=-0.3896799088561238
New Minimum: -2129.521063305694 > -2129.547719161348
END: th(0.0021802797476310636)=-2129.547719161348; dx=-1687.6188686006517 evalInputDelta=0.02665585565409856
Fitness changed from -2129.521063305694 to -2129.547719161348
Iteration 14 complete. Error: -2129.547719161348 Total: 3.9500; Orientation: 0.2986; Line Search: 3.0961
Adding measurement 5318b037 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.954e+07, gradient 2.794e-01, dot -0.166; [8b318c05-5131-4be9-a2e8-50d616ccd140 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2129.547719161348, -2129.521063305694, -404.4499688038809, -284.50626848100313, -210.53745398681443
Rejected: LBFGS Orientation magnitude: 3.558e+07, gradient 2.794e-01, dot -0.172; [8b318c05-5131-4be9-a2e8-50d616ccd140 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2129.547719161348, -2129.521063305694, -404.4499688038809, -284.50626848100313
LBFGS Accumulation History: 3 points
Removed measurement 5318b037 to history. Total: 4
Removed measurement 1389cb8e to history. Total: 3
th(0)=-2129.547719161348;dx=-0.0779066830104656
New Minimum: -2129.547719161348 > -2129.547831300498
END: th(0.004697270322270324)=-2129.547831300498; dx=-0.0628399479899161 evalInputDelta=1.1213914967811434E-4
Fitness changed from -2129.547719161348 to -2129.547831300498
Iteration 15 complete. Error: -2129.547831300498 Total: 2.3491; Orientation: 0.6356; Line Search: 1.1768
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -2129.547831300498 (> -Infinity) after 47.229s (< 5400.000s)
Returns
-2129.547831300498