BasicOptimizer.scala:89 executed in 56.39 seconds (1.785 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 223699961429700
Reset training subject: 223700795179500
Adding measurement 4a386d67 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-190.03921413526052;dx=-0.15893135139328332
New Minimum: -190.03921413526052 > -190.2622480510404
WOLFE (weak): th(2.154434690031884)=-190.2622480510404; dx=-0.15907027093279819 evalInputDelta=0.2230339157798653
New Minimum: -190.2622480510404 > -190.48590363768437
WOLFE (weak): th(4.308869380063768)=-190.48590363768437; dx=-0.1596536271948131 evalInputDelta=0.4466895024238511
New Minimum: -190.48590363768437 > -191.3821440549741
WOLFE (weak): th(12.926608140191302)=-191.3821440549741; dx=-0.15931121156792988 evalInputDelta=1.3429299197135833
New Minimum: -191.3821440549741 > -195.4844779084405
WOLFE (weak): th(51.70643256076521)=-195.4844779084405; dx=-0.15888379867052005 evalInputDelta=5.445263773179988
New Minimum: -195.4844779084405 > -219.48667449599765
WOLFE (weak): th(258.53216280382605)=-219.48667449599765; dx=-0.163646161713816 evalInputDelta=29.447460360737125
New Minimum: -219.48667449599765 > -338.2673664493135
END: th(1551.1929768229563)=-338.2673664493135; dx=-0.08842158641516973 evalInputDelta=148.22815231405298
Fitness changed from -190.03921413526052 to -338.2673664493135
Iteration 1 complete. Error: -338.2673664493135 Total: 6.6089; Orientation: 0.0165; Line Search: 4.4709
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 55b227e6 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-338.2673664493135;dx=-0.20983706467423915
New Minimum: -338.2673664493135 > -480.0155376219345
END: th(3341.943960201201)=-480.0155376219345; dx=-0.04202918323906245 evalInputDelta=141.74817117262103
Fitness changed from -338.2673664493135 to -480.0155376219345
Iteration 2 complete. Error: -480.0155376219345 Total: 1.8888; Orientation: 0.0159; Line Search: 1.2342
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 4020a1ea to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-480.0155376219345;dx=-0.19601449620907735
New Minimum: -480.0155376219345 > -601.5741395746367
END: th(7200.000000000001)=-601.5741395746367; dx=-0.018985900680879932 evalInputDelta=121.55860195270213
Fitness changed from -480.0155376219345 to -601.5741395746367
Iteration 3 complete. Error: -601.5741395746367 Total: 1.9201; Orientation: 0.0171; Line Search: 1.2834
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 1dafe9d2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.098e+03, gradient 3.729e-01, dot -0.194; [09723afb-047a-4c9a-a829-3bd6387709b4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -601.5741395746367, -480.0155376219345, -338.2673664493135, -190.03921413526052
LBFGS Accumulation History: 3 points
Removed measurement 1dafe9d2 to history. Total: 3
th(0)=-601.5741395746367;dx=-0.13784589062214453
New Minimum: -601.5741395746367 > -622.7414555643802
END: th(15511.929768229566)=-622.7414555643802; dx=-0.008209107503915142 evalInputDelta=21.167315989743543
Fitness changed from -601.5741395746367 to -622.7414555643802
Iteration 4 complete. Error: -622.7414555643802 Total: 3.8031; Orientation: 0.7350; Line Search: 2.4269
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 1f548a0b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.021e+03, gradient 3.506e-01, dot -0.164; [09723afb-047a-4c9a-a829-3bd6387709b4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -622.7414555643802, -480.0155376219345, -338.2673664493135, -190.03921413526052
LBFGS Accumulation History: 3 points
Removed measurement 1f548a0b to history. Total: 3
th(0)=-622.7414555643802;dx=-0.12280578323325472
New Minimum: -622.7414555643802 > -706.1864334409182
END: th(33419.43960201202)=-706.1864334409182; dx=-0.002283834415832438 evalInputDelta=83.44497787653802
Fitness changed from -622.7414555643802 to -706.1864334409182
Iteration 5 complete. Error: -706.1864334409182 Total: 2.9916; Orientation: 0.2958; Line Search: 1.2341
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 2c464c2d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.512e+03, gradient 2.920e-01, dot -0.122; [09723afb-047a-4c9a-a829-3bd6387709b4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -706.1864334409182, -480.0155376219345, -338.2673664493135, -190.03921413526052
LBFGS Accumulation History: 3 points
Removed measurement 2c464c2d to history. Total: 3
th(0)=-706.1864334409182;dx=-0.08519416385875474
Armijo: th(72000.00000000003)=-628.5384120501566; dx=-0.0016185990539052861 evalInputDelta=-77.64802139076164
New Minimum: -706.1864334409182 > -1039.0487292923635
END: th(36000.000000000015)=-1039.0487292923635; dx=-0.00966274952085135 evalInputDelta=332.8622958514453
Fitness changed from -706.1864334409182 to -1039.0487292923635
Iteration 6 complete. Error: -1039.0487292923635 Total: 2.9737; Orientation: 0.3742; Line Search: 2.0007
Adding measurement 2114e230 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.067e+04, gradient 3.026e-01, dot -0.135; [09723afb-047a-4c9a-a829-3bd6387709b4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1039.0487292923635, -480.0155376219345, -338.2673664493135, -190.03921413526052
LBFGS Accumulation History: 3 points
Removed measurement 2114e230 to history. Total: 3
th(0)=-1039.0487292923635;dx=-0.09131501343826437
Armijo: th(77559.64884114785)=-838.1956784338463; dx=-5.984292258271643E-4 evalInputDelta=-200.85305085851724
New Minimum: -1039.0487292923635 > -1422.8751897186858
END: th(38779.824420573925)=-1422.8751897186858; dx=-0.009458561320399024 evalInputDelta=383.8264604263222
Fitness changed from -1039.0487292923635 to -1422.8751897186858
Iteration 7 complete. Error: -1422.8751897186858 Total: 3.4991; Orientation: 0.5210; Line Search: 2.2559
Adding measurement 70f0f86b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.224e+04, gradient 2.892e-01, dot -0.163; [09723afb-047a-4c9a-a829-3bd6387709b4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1422.8751897186858, -480.0155376219345, -338.2673664493135, -190.03921413526052
LBFGS Accumulation History: 3 points
Removed measurement 70f0f86b to history. Total: 3
th(0)=-1422.8751897186858;dx=-0.08328960437055943
Armijo: th(83548.59900503006)=-1058.456707806028; dx=4.2581125672367656E-4 evalInputDelta=-364.41848191265785
New Minimum: -1422.8751897186858 > -1784.4573956932643
END: th(41774.29950251503)=-1784.4573956932643; dx=-0.010369004874372027 evalInputDelta=361.5822059745785
Fitness changed from -1422.8751897186858 to -1784.4573956932643
Iteration 8 complete. Error: -1784.4573956932643 Total: 3.2229; Orientation: 0.3193; Line Search: 2.3271
Adding measurement 1f8b26ee to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.599e+04, gradient 2.972e-01, dot -0.174; [09723afb-047a-4c9a-a829-3bd6387709b4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1784.4573956932643, -480.0155376219345, -338.2673664493135, -190.03921413526052
LBFGS Accumulation History: 3 points
Removed measurement 1f8b26ee to history. Total: 3
th(0)=-1784.4573956932643;dx=-0.08810906942608202
Armijo: th(90000.00000000004)=-1186.0341092158574; dx=0.001977260363819465 evalInputDelta=-598.4232864774069
New Minimum: -1784.4573956932643 > -2095.1151124239227
END: th(45000.00000000002)=-2095.1151124239227; dx=-0.008162773169739346 evalInputDelta=310.6577167306584
Fitness changed from -1784.4573956932643 to -2095.1151124239227
Iteration 9 complete. Error: -2095.1151124239227 Total: 2.6717; Orientation: 0.2865; Line Search: 1.8230
Adding measurement 83c6a07 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.163e+05, gradient 3.024e-01, dot -0.179; [09723afb-047a-4c9a-a829-3bd6387709b4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2095.1151124239227, -480.0155376219345, -338.2673664493135, -190.03921413526052
LBFGS Accumulation History: 3 points
Removed measurement 83c6a07 to history. Total: 3
th(0)=-2095.1151124239227;dx=-0.09123539343981867
Armijo: th(96949.56105143482)=-1285.8779450772208; dx=0.0024476946022392943 evalInputDelta=-809.2371673467019
New Minimum: -2095.1151124239227 > -2346.517582147944
END: th(48474.78052571741)=-2346.517582147944; dx=-0.006357332763322409 evalInputDelta=251.4024697240211
Fitness changed from -2095.1151124239227 to -2346.517582147944
Iteration 10 complete. Error: -2346.517582147944 Total: 2.7270; Orientation: 0.3234; Line Search: 1.8570
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 24312981 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.838e+06, gradient 3.066e-01, dot -0.183; [09723afb-047a-4c9a-a829-3bd6387709b4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2346.517582147944, -480.0155376219345, -338.2673664493135, -190.03921413526052
LBFGS Accumulation History: 3 points
Removed measurement 24312981 to history. Total: 3
th(0)=-2346.517582147944;dx=-0.09390799609600554
Armijo: th(104435.74875628759)=-1464.46741575694; dx=0.002748025196482274 evalInputDelta=-882.0501663910038
New Minimum: -2346.517582147944 > -2604.8579714658968
END: th(52217.874378143795)=-2604.8579714658968; dx=-0.005165938914493545 evalInputDelta=258.34038931795294
Fitness changed from -2346.517582147944 to -2604.8579714658968
Iteration 11 complete. Error: -2604.8579714658968 Total: 2.7342; Orientation: 0.3057; Line Search: 1.8387
Adding measurement 4ef6ea2c to history. Total: 3
Accepted: LBFGS Orientation magnitude: 7.401e+04, gradient 3.031e-01, dot 0.203; [09723afb-047a-4c9a-a829-3bd6387709b4 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-2604.8579714658968;dx=-4564.125577804751
Armijo: th(112500.00000000007)=-1293.5024381421345; dx=0.010442469475479495 evalInputDelta=-1311.3555333237623
Armijo: th(56250.00000000004)=-1293.8004580278748; dx=0.02085888781404346 evalInputDelta=-1311.057513438022
Armijo: th(18750.00000000001)=-1294.5370444349273; dx=0.06261087760743617 evalInputDelta=-1310.3209270309694
Armijo: th(4687.500000000003)=-1297.4380918436455; dx=0.24995049683641915 evalInputDelta=-1307.4198796222513
Armijo: th(937.5000000000006)=-1307.3936686186112; dx=1.2347022164978485 evalInputDelta=-1297.4643028472856
Armijo: th(156.25000000000009)=-1335.7042454321868; dx=7.19376892906173 evalInputDelta=-1269.15372603371
Armijo: th(22.321428571428584)=-1428.2776739099095; dx=44.48508285315149 evalInputDelta=-1176.5802975559873
Armijo: th(2.790178571428573)=-1775.5067715174446; dx=203.0200459514999 evalInputDelta=-829.3511999484522
Armijo: th(0.31001984126984145)=-2490.6530440674483; dx=-106.16771577414409 evalInputDelta=-114.20492739844849
New Minimum: -2604.8579714658968 > -2621.676167835917
END: th(0.031001984126984145)=-2621.676167835917; dx=-1360.628787744859 evalInputDelta=16.81819637002036
Fitness changed from -2604.8579714658968 to -2621.676167835917
Iteration 12 complete. Error: -2621.676167835917 Total: 8.6844; Orientation: 0.3036; Line Search: 7.7900
Adding measurement e99e1b2 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 3.632e+05, gradient 2.996e-01, dot -0.143; [09723afb-047a-4c9a-a829-3bd6387709b4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2621.676167835917, -2604.8579714658968, -480.0155376219345, -338.2673664493135, -190.03921413526052
Rejected: LBFGS Orientation magnitude: 1.011e+06, gradient 2.996e-01, dot -0.161; [09723afb-047a-4c9a-a829-3bd6387709b4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2621.676167835917, -2604.8579714658968, -480.0155376219345, -338.2673664493135
LBFGS Accumulation History: 3 points
Removed measurement e99e1b2 to history. Total: 4
Removed measurement 4ef6ea2c to history. Total: 3
th(0)=-2621.676167835917;dx=-0.08959565733400213
New Minimum: -2621.676167835917 > -2621.6791787029492
END: th(0.06679175006299247)=-2621.6791787029492; dx=-0.06556535574470583 evalInputDelta=0.0030108670321169484
Fitness changed from -2621.676167835917 to -2621.6791787029492
Iteration 13 complete. Error: -2621.6791787029492 Total: 2.5249; Orientation: 0.6861; Line Search: 1.2369
Adding measurement 3f0588f4 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 5.188e+04, gradient 2.995e-01, dot 0.211; [09723afb-047a-4c9a-a829-3bd6387709b4 = 1.000/1.000e+00]
th(0)=-2621.6791787029492;dx=-3278.4927888393486
Armijo: th(0.14389846334365022)=-2614.256919997669; dx=-595.6914789812208 evalInputDelta=-7.422258705280456
New Minimum: -2621.6791787029492 > -2628.8480601781794
END: th(0.07194923167182511)=-2628.8480601781794; dx=-829.875316432568 evalInputDelta=7.168881475230137
Fitness changed from -2621.6791787029492 to -2628.8480601781794
Iteration 14 complete. Error: -2628.8480601781794 Total: 2.9881; Orientation: 0.2976; Line Search: 2.1459
Adding measurement 1956f837 to history. Total: 4
Accepted: LBFGS Orientation magnitude: 4.858e+04, gradient 2.942e-01, dot 0.217; [09723afb-047a-4c9a-a829-3bd6387709b4 = 1.000/1.000e+00]
th(0)=-2628.8480601781794;dx=-3097.296165095623
Armijo: th(0.15500992063492072)=-2590.8964618584737; dx=-409.56321924055 evalInputDelta=-37.95159831970568
Armijo: th(0.07750496031746036)=-2614.8638860800597; dx=-545.4714723677967 evalInputDelta=-13.984174098119638
Armijo: th(0.025834986772486787)=-2625.750791616839; dx=-660.171617278542 evalInputDelta=-3.09726856134057
Armijo: th(0.006458746693121697)=-2628.2728949303983; dx=-703.7106841088691 evalInputDelta=-0.5751652477811149
Armijo: th(0.0012917493386243393)=-2628.7399590509303; dx=-716.2637743577865 evalInputDelta=-0.10810112724902865
Armijo: th(2.152915564373899E-4)=-2628.830360178455; dx=-719.7768041162336 evalInputDelta=-0.01769999972430014
Armijo: th(3.0755936633912844E-5)=-2628.845495936422; dx=-720.3625559793115 evalInputDelta=-0.002564241757227137
Armijo: th(3.8444920792391055E-6)=-2628.847873137532; dx=-719.1978568477031 evalInputDelta=-1.870406472335162E-4
New Minimum: -2628.8480601781794 > -2628.8481726716022
END: th(4.2716578658212286E-7)=-2628.8481726716022; dx=-718.6109586599275 evalInputDelta=1.1249342287555919E-4
Fitness changed from -2628.8480601781794 to -2628.8481726716022
Iteration 15 complete. Error: -2628.8481726716022 Total: 7.1290; Orientation: 0.4008; Line Search: 6.1443
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -2628.8481726716022 (> -Infinity) after 56.391s (< 5400.000s)
Returns
-2628.8481726716022