BasicOptimizer.scala:89 executed in 45.00 seconds (0.842 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 214293997311000
Reset training subject: 214294599068200
Adding measurement 2396e6e1 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-18.477675660145522;dx=-0.007729256311412873
New Minimum: -18.477675660145522 > -18.487433776988137
WOLFE (weak): th(2.154434690031884)=-18.487433776988137; dx=-0.00774760928845002 evalInputDelta=0.009758116842615294
New Minimum: -18.487433776988137 > -18.497192955320525
WOLFE (weak): th(4.308869380063768)=-18.497192955320525; dx=-0.007747614714858264 evalInputDelta=0.019517295175003113
New Minimum: -18.497192955320525 > -18.536343316387637
WOLFE (weak): th(12.926608140191302)=-18.536343316387637; dx=-0.007761466060806764 evalInputDelta=0.058667656242114674
New Minimum: -18.536343316387637 > -18.712726654298994
WOLFE (weak): th(51.70643256076521)=-18.712726654298994; dx=-0.0077150525793654254 evalInputDelta=0.23505099415347175
New Minimum: -18.712726654298994 > -19.67118279851048
WOLFE (weak): th(258.53216280382605)=-19.67118279851048; dx=-0.007929456234251156 evalInputDelta=1.1935071383649571
New Minimum: -19.67118279851048 > -26.230030686133198
WOLFE (weak): th(1551.1929768229563)=-26.230030686133198; dx=-0.00760293075883121 evalInputDelta=7.752355025987676
New Minimum: -26.230030686133198 > -50.860113025648275
END: th(10858.350837760694)=-50.860113025648275; dx=-0.0017535072246125942 evalInputDelta=32.38243736550275
Fitness changed from -18.477675660145522 to -50.860113025648275
Iteration 1 complete. Error: -50.860113025648275 Total: 6.7320; Orientation: 0.0160; Line Search: 4.9815
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 7d957c96 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-50.860113025648275;dx=-0.016230307828415926
New Minimum: -50.860113025648275 > -91.39438760995277
END: th(23393.607721408407)=-91.39438760995277; dx=-5.551630049757192E-4 evalInputDelta=40.5342745843045
Fitness changed from -50.860113025648275 to -91.39438760995277
Iteration 2 complete. Error: -91.39438760995277 Total: 1.8192; Orientation: 0.0173; Line Search: 1.2259
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 4c6e3c25 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-91.39438760995277;dx=-0.011917088947028722
New Minimum: -91.39438760995277 > -114.84339466843186
END: th(50400.00000000001)=-114.84339466843186; dx=-4.39601270551478E-4 evalInputDelta=23.449007058479083
Fitness changed from -91.39438760995277 to -114.84339466843186
Iteration 3 complete. Error: -114.84339466843186 Total: 2.0349; Orientation: 0.0158; Line Search: 1.2337
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 36d61bbf to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.356e+03, gradient 1.035e-01, dot -0.206; [67a16201-85c2-47c4-ade2-45004a3c8861 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -114.84339466843186, -91.39438760995277, -50.860113025648275, -18.477675660145522
LBFGS Accumulation History: 3 points
Removed measurement 36d61bbf to history. Total: 3
th(0)=-114.84339466843186;dx=-0.010689070838134841
New Minimum: -114.84339466843186 > -136.63112623707667
END: th(108583.50837760697)=-136.63112623707667; dx=-3.122651558319843E-4 evalInputDelta=21.787731568644816
Fitness changed from -114.84339466843186 to -136.63112623707667
Iteration 4 complete. Error: -136.63112623707667 Total: 2.0561; Orientation: 0.2967; Line Search: 1.2001
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 7fabe964 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.733e+03, gradient 9.095e-02, dot -0.161; [67a16201-85c2-47c4-ade2-45004a3c8861 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -136.63112623707667, -91.39438760995277, -50.860113025648275, -18.477675660145522
LBFGS Accumulation History: 3 points
Removed measurement 7fabe964 to history. Total: 3
th(0)=-136.63112623707667;dx=-0.008252197674981972
Armijo: th(233936.07721408413)=-95.47323477895316; dx=-1.1827515702525941E-4 evalInputDelta=-41.157891458123515
New Minimum: -136.63112623707667 > -188.83520322885116
END: th(116968.03860704206)=-188.83520322885116; dx=-6.563491838200829E-4 evalInputDelta=52.204076991774485
Fitness changed from -136.63112623707667 to -188.83520322885116
Iteration 5 complete. Error: -188.83520322885116 Total: 2.7253; Orientation: 0.2949; Line Search: 1.8311
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 35e55526 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.227e+03, gradient 8.641e-02, dot -0.128; [67a16201-85c2-47c4-ade2-45004a3c8861 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -188.83520322885116, -91.39438760995277, -50.860113025648275, -18.477675660145522
LBFGS Accumulation History: 3 points
Removed measurement 35e55526 to history. Total: 3
th(0)=-188.83520322885116;dx=-0.007463379893003882
Armijo: th(252000.0000000001)=-178.61108053211504; dx=-9.80003011095767E-5 evalInputDelta=-10.224122696736117
New Minimum: -188.83520322885116 > -300.5249994133617
END: th(126000.00000000004)=-300.5249994133617; dx=-8.933744694828319E-4 evalInputDelta=111.68979618451053
Fitness changed from -188.83520322885116 to -300.5249994133617
Iteration 6 complete. Error: -300.5249994133617 Total: 2.6991; Orientation: 0.3135; Line Search: 1.7894
Adding measurement 2a4d39fa to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.327e+04, gradient 8.172e-02, dot -0.126; [67a16201-85c2-47c4-ade2-45004a3c8861 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -300.5249994133617, -91.39438760995277, -50.860113025648275, -18.477675660145522
LBFGS Accumulation History: 3 points
Removed measurement 2a4d39fa to history. Total: 3
th(0)=-300.5249994133617;dx=-0.006672437114778367
Armijo: th(271458.77094401745)=-233.78259914341294; dx=-1.063394790260206E-4 evalInputDelta=-66.74240026994875
New Minimum: -300.5249994133617 > -391.6226770958047
END: th(135729.38547200873)=-391.6226770958047; dx=-9.978289683886623E-4 evalInputDelta=91.09767768244302
Fitness changed from -300.5249994133617 to -391.6226770958047
Iteration 7 complete. Error: -391.6226770958047 Total: 2.9596; Orientation: 0.2944; Line Search: 2.1229
Adding measurement 55307965 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.935e+04, gradient 8.621e-02, dot -0.142; [67a16201-85c2-47c4-ade2-45004a3c8861 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -391.6226770958047, -91.39438760995277, -50.860113025648275, -18.477675660145522
LBFGS Accumulation History: 3 points
Removed measurement 55307965 to history. Total: 3
th(0)=-391.6226770958047;dx=-0.007424115308269623
Armijo: th(292420.0965176052)=-266.74806749064015; dx=-7.38829363632132E-5 evalInputDelta=-124.87460960516455
New Minimum: -391.6226770958047 > -459.5341077843394
END: th(146210.0482588026)=-459.5341077843394; dx=-8.991893378125296E-4 evalInputDelta=67.91143068853472
Fitness changed from -391.6226770958047 to -459.5341077843394
Iteration 8 complete. Error: -459.5341077843394 Total: 2.6320; Orientation: 0.2816; Line Search: 1.8061
Adding measurement 4a6898d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.430e+05, gradient 8.925e-02, dot -0.141; [67a16201-85c2-47c4-ade2-45004a3c8861 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -459.5341077843394, -91.39438760995277, -50.860113025648275, -18.477675660145522
LBFGS Accumulation History: 3 points
Removed measurement 4a6898d to history. Total: 3
th(0)=-459.5341077843394;dx=-0.007940997549551687
Armijo: th(315000.0000000001)=-246.3881429553977; dx=4.314041088271156E-5 evalInputDelta=-213.14596482894171
New Minimum: -459.5341077843394 > -495.6675925662428
END: th(157500.00000000006)=-495.6675925662428; dx=-7.296752228752885E-4 evalInputDelta=36.13348478190335
Fitness changed from -459.5341077843394 to -495.6675925662428
Iteration 9 complete. Error: -495.6675925662428 Total: 2.6257; Orientation: 0.2929; Line Search: 1.7769
Adding measurement 16a1738d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.031e+04, gradient 9.203e-02, dot -0.108; [67a16201-85c2-47c4-ade2-45004a3c8861 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -495.6675925662428, -91.39438760995277, -50.860113025648275, -18.477675660145522
LBFGS Accumulation History: 3 points
Removed measurement 16a1738d to history. Total: 3
th(0)=-495.6675925662428;dx=-0.008461635941570317
Armijo: th(339323.4636800218)=-251.0171953947281; dx=1.364155564889603E-4 evalInputDelta=-244.65039717151467
New Minimum: -495.6675925662428 > -525.9644950228223
END: th(169661.7318400109)=-525.9644950228223; dx=-5.54531304750352E-4 evalInputDelta=30.296902456579573
Fitness changed from -495.6675925662428 to -525.9644950228223
Iteration 10 complete. Error: -525.9644950228223 Total: 2.7178; Orientation: 0.3019; Line Search: 1.8815
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement f284a7a to history. Total: 3
Accepted: LBFGS Orientation magnitude: 3.523e+04, gradient 9.415e-02, dot 0.148; [67a16201-85c2-47c4-ade2-45004a3c8861 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-525.9644950228223;dx=-489.39101761749345
Armijo: th(365525.1206470065)=313.6767959109826; dx=0.0015297050159192604 evalInputDelta=-839.641290933805
Armijo: th(182762.56032350325)=311.3091570613861; dx=0.0030542202214626996 evalInputDelta=-837.2736520842084
Armijo: th(60920.85344116775)=305.33401786520415; dx=0.009086549995409622 evalInputDelta=-831.2985128880265
Armijo: th(15230.213360291937)=290.8508551641037; dx=0.03560300349246852 evalInputDelta=-816.815350186926
Armijo: th(3046.0426720583873)=259.4949202689798; dx=0.1690820700072556 evalInputDelta=-785.4594152918021
Armijo: th(507.6737786763979)=196.82426839032996; dx=0.9121165578846817 evalInputDelta=-722.7887634131523
Armijo: th(72.5248255251997)=82.8491098699626; dx=5.119670458257319 evalInputDelta=-608.813604892785
Armijo: th(9.065603190649963)=-108.92821792498364; dx=24.057972634843136 evalInputDelta=-417.0362770978387
Armijo: th(1.0072892434055514)=-401.3987548383707; dx=33.7197422103806 evalInputDelta=-124.56574018445167
Armijo: th(0.10072892434055514)=-520.3640456125689; dx=-100.36164724453496 evalInputDelta=-5.600449410253418
Armijo: th(0.009157174940050467)=-525.7937344906567; dx=-153.13121068484497 evalInputDelta=-0.17076053216567288
Armijo: th(7.630979116708723E-4)=-525.9530799787228; dx=-158.6594774736114 evalInputDelta=-0.011415044099521765
Armijo: th(5.869983935929787E-5)=-525.9636557784492; dx=-159.23973726985665 evalInputDelta=-8.392443731963795E-4
Armijo: th(4.192845668521276E-6)=-525.9644354058727; dx=-159.73091426410156 evalInputDelta=-5.9616949670271424E-5
New Minimum: -525.9644950228223 > -525.9645091282458
END: th(2.795230445680851E-7)=-525.9645091282458; dx=-159.456580259661 evalInputDelta=1.4105423474575218E-5
Fitness changed from -525.9644950228223 to -525.9645091282458
Iteration 11 complete. Error: -525.9645091282458 Total: 10.9666; Orientation: 0.2904; Line Search: 10.1071
Adding measurement 10ecd658 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.985e+06, gradient 9.413e-02, dot -0.013; [67a16201-85c2-47c4-ade2-45004a3c8861 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -525.9645091282458, -525.9644950228223, -91.39438760995277, -50.860113025648275, -18.477675660145522
Rejected: LBFGS Orientation magnitude: 7.677e+06, gradient 9.413e-02, dot -0.015; [67a16201-85c2-47c4-ade2-45004a3c8861 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -525.9645091282458, -525.9644950228223, -91.39438760995277, -50.860113025648275
LBFGS Accumulation History: 3 points
Removed measurement 10ecd658 to history. Total: 4
Removed measurement f284a7a to history. Total: 3
th(0)=-525.9645091282458;dx=-0.0088452689543182
END: th(6.022141438808108E-7)=-525.9645091282458; dx=-0.006905627104857384 evalInputDelta=0.0
Fitness changed from -525.9645091282458 to -525.9645091282458
Static Iteration Total: 2.6872; Orientation: 0.6520; Line Search: 1.4789
Iteration 12 failed. Error: -525.9645091282458
Previous Error: 0.0 -> -525.9645091282458
Retrying iteration 12
Reset training subject: 214336652838800
Adding measurement 691a1521 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-525.9645091282458;dx=-0.008843128610313576
END: th(1.2974310424046709E-6)=-525.9645091282458; dx=-0.006892241757651856 evalInputDelta=0.0
Fitness changed from -525.9645091282458 to -525.9645091282458
Static Iteration Total: 2.3413; Orientation: 0.0159; Line Search: 1.2214
Iteration 13 failed. Error: -525.9645091282458
Previous Error: 0.0 -> -525.9645091282458
Optimization terminated 13
Final threshold in iteration 13: -525.9645091282458 (> -Infinity) after 44.997s (< 5400.000s)
Returns
-525.9645091282458