BasicOptimizer.scala:89 executed in 244.00 seconds (2.871 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 5643895021871
Reset training subject: 5645485084709
Adding measurement 7eb712df to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-2.3794783418060623;dx=-2.0749812847659275E-4
New Minimum: -2.3794783418060623 > -2.3795076126432666
WOLFE (weak): th(2.154434690031884)=-2.3795076126432666; dx=-2.0846817299907607E-4 evalInputDelta=2.9270837204276035E-5
New Minimum: -2.3795076126432666 > -2.3795344991131144
WOLFE (weak): th(4.308869380063768)=-2.3795344991131144; dx=-2.0726740419971105E-4 evalInputDelta=5.6157307052107086E-5
New Minimum: -2.3795344991131144 > -2.379653299237191
WOLFE (weak): th(12.926608140191302)=-2.379653299237191; dx=-2.0884572685979128E-4 evalInputDelta=1.7495743112849738E-4
New Minimum: -2.379653299237191 > -2.380190540791183
WOLFE (weak): th(51.70643256076521)=-2.380190540791183; dx=-2.0776548733238286E-4 evalInputDelta=7.12198985120871E-4
New Minimum: -2.380190540791183 > -2.3832431129722114
WOLFE (weak): th(258.53216280382605)=-2.3832431129722114; dx=-2.0736725871088542E-4 evalInputDelta=0.003764771166149039
New Minimum: -2.3832431129722114 > -2.401495698628983
WOLFE (weak): th(1551.1929768229563)=-2.401495698628983; dx=-2.014470927595934E-4 evalInputDelta=0.022017356822920497
New Minimum: -2.401495698628983 > -2.4893841374903607
END: th(10858.350837760694)=-2.4893841374903607; dx=-1.61594138519889E-4 evalInputDelta=0.10990579568429837
Fitness changed from -2.3794783418060623 to -2.4893841374903607
Iteration 1 complete. Error: -2.4893841374903607 Total: 21.6757; Orientation: 0.0186; Line Search: 11.1387
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 4510c549 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-2.4893841374903607;dx=-2.1560168039117607E-4
New Minimum: -2.4893841374903607 > -2.7981517445398154
END: th(23393.607721408407)=-2.7981517445398154; dx=-1.3363702295873412E-4 evalInputDelta=0.3087676070494547
Fitness changed from -2.4893841374903607 to -2.7981517445398154
Iteration 2 complete. Error: -2.7981517445398154 Total: 4.3435; Orientation: 0.0181; Line Search: 2.7096
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 2c928732 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-2.7981517445398154;dx=-2.6187851370141593E-4
New Minimum: -2.7981517445398154 > -3.9764513324652593
END: th(50400.00000000001)=-3.9764513324652593; dx=-6.269927949030462E-5 evalInputDelta=1.1782995879254439
Fitness changed from -2.7981517445398154 to -3.9764513324652593
Iteration 3 complete. Error: -3.9764513324652593 Total: 4.6070; Orientation: 0.0288; Line Search: 2.9743
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 11f0b86e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.603e+03, gradient 1.631e-02, dot -0.209; [6f698610-1f57-4f4d-bcb3-ea51c1524cb3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.9764513324652593, -2.7981517445398154, -2.4893841374903607, -2.3794783418060623
LBFGS Accumulation History: 3 points
Removed measurement 11f0b86e to history. Total: 3
th(0)=-3.9764513324652593;dx=-2.616157870556459E-4
Armijo: th(108583.50837760697)=-3.915661676526574; dx=-2.2896478440372206E-5 evalInputDelta=-0.06078965593868535
New Minimum: -3.9764513324652593 > -4.25611752442166
END: th(54291.75418880348)=-4.25611752442166; dx=-7.118810788680203E-5 evalInputDelta=0.2796661919564003
Fitness changed from -3.9764513324652593 to -4.25611752442166
Iteration 4 complete. Error: -4.25611752442166 Total: 6.1479; Orientation: 0.5091; Line Search: 4.0329
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 3dd9e647 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.871e+03, gradient 1.621e-02, dot -0.199; [6f698610-1f57-4f4d-bcb3-ea51c1524cb3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.25611752442166, -2.7981517445398154, -2.4893841374903607, -2.3794783418060623
LBFGS Accumulation History: 3 points
Removed measurement 3dd9e647 to history. Total: 3
th(0)=-4.25611752442166;dx=-2.5846418615760287E-4
New Minimum: -4.25611752442166 > -4.386015738564412
END: th(116968.03860704206)=-4.386015738564412; dx=-2.2751206530936436E-5 evalInputDelta=0.12989821414275227
Fitness changed from -4.25611752442166 to -4.386015738564412
Iteration 5 complete. Error: -4.386015738564412 Total: 4.9168; Orientation: 0.5151; Line Search: 2.6392
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 7519e1d4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.091e+03, gradient 1.329e-02, dot -0.114; [6f698610-1f57-4f4d-bcb3-ea51c1524cb3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.386015738564412, -2.7981517445398154, -2.4893841374903607, -2.3794783418060623
LBFGS Accumulation History: 3 points
Removed measurement 7519e1d4 to history. Total: 3
th(0)=-4.386015738564412;dx=-1.7149670128710162E-4
Armijo: th(252000.0000000001)=-4.272151899276496; dx=-4.097326396085559E-6 evalInputDelta=-0.11386383928791588
New Minimum: -4.386015738564412 > -4.980995945203784
END: th(126000.00000000004)=-4.980995945203784; dx=-2.4795073026688025E-5 evalInputDelta=0.5949802066393719
Fitness changed from -4.386015738564412 to -4.980995945203784
Iteration 6 complete. Error: -4.980995945203784 Total: 6.2837; Orientation: 0.4985; Line Search: 4.2105
Adding measurement 344e49fd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.436e+03, gradient 1.244e-02, dot -0.114; [6f698610-1f57-4f4d-bcb3-ea51c1524cb3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.980995945203784, -2.7981517445398154, -2.4893841374903607, -2.3794783418060623
LBFGS Accumulation History: 3 points
Removed measurement 344e49fd to history. Total: 3
th(0)=-4.980995945203784;dx=-1.5272524680745285E-4
Armijo: th(271458.77094401745)=-4.565042066591605; dx=-7.831981709670568E-6 evalInputDelta=-0.4159538786121786
New Minimum: -4.980995945203784 > -5.1240360261269275
END: th(135729.38547200873)=-5.1240360261269275; dx=-2.3788787180572767E-5 evalInputDelta=0.1430400809231438
Fitness changed from -4.980995945203784 to -5.1240360261269275
Iteration 7 complete. Error: -5.1240360261269275 Total: 5.7830; Orientation: 0.4967; Line Search: 3.9870
Adding measurement 1bb977af to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.248e+03, gradient 1.112e-02, dot -0.111; [6f698610-1f57-4f4d-bcb3-ea51c1524cb3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.1240360261269275, -2.7981517445398154, -2.4893841374903607, -2.3794783418060623
LBFGS Accumulation History: 3 points
Removed measurement 1bb977af to history. Total: 3
th(0)=-5.1240360261269275;dx=-1.202369601939233E-4
Armijo: th(292420.0965176052)=-4.678873377824371; dx=-5.563161976180269E-6 evalInputDelta=-0.44516264830255636
New Minimum: -5.1240360261269275 > -5.323417913435582
END: th(146210.0482588026)=-5.323417913435582; dx=-2.2211018962090383E-5 evalInputDelta=0.1993818873086548
Fitness changed from -5.1240360261269275 to -5.323417913435582
Iteration 8 complete. Error: -5.323417913435582 Total: 5.9165; Orientation: 0.5065; Line Search: 4.0244
Adding measurement 3274b0b0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.138e+03, gradient 9.873e-03, dot -0.108; [6f698610-1f57-4f4d-bcb3-ea51c1524cb3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.323417913435582, -2.7981517445398154, -2.4893841374903607, -2.3794783418060623
LBFGS Accumulation History: 3 points
Removed measurement 3274b0b0 to history. Total: 3
th(0)=-5.323417913435582;dx=-9.583124259641737E-5
New Minimum: -5.323417913435582 > -5.832267372850638
END: th(315000.0000000001)=-5.832267372850638; dx=-9.682278667839441E-6 evalInputDelta=0.5088494594150559
Fitness changed from -5.323417913435582 to -5.832267372850638
Iteration 9 complete. Error: -5.83226
...skipping 17025 bytes...
4123.009259359)=-10.31533581607648; dx=-4.3038018854825955E-6 evalInputDelta=0.3601872376571116
Fitness changed from -9.955148578419369 to -10.31533581607648
Iteration 28 complete. Error: -10.31533581607648 Total: 5.8813; Orientation: 0.4791; Line Search: 4.1097
Adding measurement 16b8f265 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.547e+05, gradient 3.660e-03, dot -0.203; [6f698610-1f57-4f4d-bcb3-ea51c1524cb3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.31533581607648, -2.7981517445398154, -2.4893841374903607, -2.3794783418060623
LBFGS Accumulation History: 3 points
Removed measurement 16b8f265 to history. Total: 3
th(0)=-10.31533581607648;dx=-1.300108630768162E-5
Armijo: th(3305167.829924468)=-8.45300974792219; dx=-1.2687464856708744E-6 evalInputDelta=-1.8623260681542906
Armijo: th(1652583.914962234)=-9.932564434105908; dx=-3.870609258395044E-6 evalInputDelta=-0.3827713819705725
New Minimum: -10.31533581607648 > -10.49145797923614
END: th(550861.3049874114)=-10.49145797923614; dx=-8.055869290775327E-6 evalInputDelta=0.17612216315965945
Fitness changed from -10.31533581607648 to -10.49145797923614
Iteration 29 complete. Error: -10.49145797923614 Total: 7.5445; Orientation: 0.4840; Line Search: 5.7915
Adding measurement 23dcf288 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.380e+05, gradient 3.636e-03, dot -0.220; [6f698610-1f57-4f4d-bcb3-ea51c1524cb3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.49145797923614, -2.7981517445398154, -2.4893841374903607, -2.3794783418060623
LBFGS Accumulation History: 3 points
Removed measurement 23dcf288 to history. Total: 3
th(0)=-10.49145797923614;dx=-1.2903934494389279E-5
Armijo: th(1186794.7048611126)=-10.274667036681265; dx=-5.076515001171139E-6 evalInputDelta=-0.21679094255487463
New Minimum: -10.49145797923614 > -10.545095265299825
END: th(593397.3524305563)=-10.545095265299825; dx=-7.498551419066878E-6 evalInputDelta=0.0536372860636849
Fitness changed from -10.49145797923614 to -10.545095265299825
Iteration 30 complete. Error: -10.545095265299825 Total: 6.1493; Orientation: 0.5224; Line Search: 4.1295
<a id="p-21"></a>Iteration 30
<a id="p-20"></a>
Adding measurement fca8474 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.655e+05, gradient 3.415e-03, dot -0.236; [6f698610-1f57-4f4d-bcb3-ea51c1524cb3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.545095265299825, -2.7981517445398154, -2.4893841374903607, -2.3794783418060623
LBFGS Accumulation History: 3 points
Removed measurement fca8474 to history. Total: 3
th(0)=-10.545095265299825;dx=-1.160867763683064E-5
Armijo: th(1278435.841049466)=-10.408564413737064; dx=-4.645342084254628E-6 evalInputDelta=-0.13653085156276035
Armijo: th(639217.920524733)=-10.477626886975452; dx=-6.969150617592435E-6 evalInputDelta=-0.06746837832437258
New Minimum: -10.545095265299825 > -10.55055569098714
END: th(213072.640174911)=-10.55055569098714; dx=-9.0247774745901E-6 evalInputDelta=0.005460425687315862
Fitness changed from -10.545095265299825 to -10.55055569098714
Iteration 31 complete. Error: -10.55055569098714 Total: 10.0180; Orientation: 0.6350; Line Search: 7.7719
Adding measurement 53a221e2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.052e+06, gradient 3.609e-03, dot -0.234; [6f698610-1f57-4f4d-bcb3-ea51c1524cb3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.55055569098714, -2.7981517445398154, -2.4893841374903607, -2.3794783418060623
LBFGS Accumulation History: 3 points
Removed measurement 53a221e2 to history. Total: 3
th(0)=-10.55055569098714;dx=-1.2605803989129201E-5
New Minimum: -10.55055569098714 > -10.584085710405192
END: th(459051.0874895095)=-10.584085710405192; dx=-8.495993608850327E-6 evalInputDelta=0.03353001941805189
Fitness changed from -10.55055569098714 to -10.584085710405192
Iteration 32 complete. Error: -10.584085710405192 Total: 4.8284; Orientation: 0.5049; Line Search: 2.7083
Adding measurement 58dc62c4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.230e+05, gradient 3.635e-03, dot -0.244; [6f698610-1f57-4f4d-bcb3-ea51c1524cb3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.584085710405192, -2.7981517445398154, -2.4893841374903607, -2.3794783418060623
LBFGS Accumulation History: 3 points
Removed measurement 58dc62c4 to history. Total: 3
th(0)=-10.584085710405192;dx=-1.2817383545824596E-5
Armijo: th(988995.5873842606)=-10.55152490559143; dx=-6.138820934121917E-6 evalInputDelta=-0.032560804813762445
New Minimum: -10.584085710405192 > -10.650495305424895
END: th(494497.7936921303)=-10.650495305424895; dx=-8.218390821010748E-6 evalInputDelta=0.066409595019703
Fitness changed from -10.584085710405192 to -10.650495305424895
Iteration 33 complete. Error: -10.650495305424895 Total: 5.9557; Orientation: 0.4817; Line Search: 4.1977
Adding measurement 46c6bf50 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 3.518e+04, gradient 3.636e-03, dot 0.262; [6f698610-1f57-4f4d-bcb3-ea51c1524cb3 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-10.650495305424895;dx=-33.393129051559555
Armijo: th(1065363.2008745552)=-7.3684167876244695; dx=1.7421567242156E-6 evalInputDelta=-3.282078517800426
Armijo: th(532681.6004372776)=-7.3684167876244695; dx=3.4532670646727436E-6 evalInputDelta=-3.282078517800426
Armijo: th(177560.53347909253)=-7.3684167876244695; dx=1.0256029280677144E-5 evalInputDelta=-3.282078517800426
Armijo: th(44390.13336977313)=-7.369577340000583; dx=4.044766850482108E-5 evalInputDelta=-3.2809179654243126
Armijo: th(8878.026673954626)=-7.361531144423591; dx=2.0734687094356083E-4 evalInputDelta=-3.2889641610013047
Armijo: th(1479.671112325771)=-7.378311202279874; dx=0.0011777429627058147 evalInputDelta=-3.2721841031450216
Armijo: th(211.38158747511014)=-7.4836862996466795; dx=0.008600024446599838 evalInputDelta=-3.166809005778216
Armijo: th(26.422698434388767)=-7.375646050232276; dx=0.05247765635266097 evalInputDelta=-3.2748492551926196
Armijo: th(2.935855381598752)=-8.52535159091103; dx=-0.5775206600953363 evalInputDelta=-2.1251437145138663
Armijo: th(0.2935855381598752)=-10.133434796131281; dx=-13.69746435659981 evalInputDelta=-0.5170605092936142
Armijo: th(0.026689594378170475)=-10.604119925903573; dx=-22.25810295063864 evalInputDelta=-0.0463753795213222
Armijo: th(0.0022241328648475397)=-10.645736032810198; dx=-23.881953778615426 evalInputDelta=-0.004759272614696997
Armijo: th(1.7108714344981076E-4)=-10.650126952109952; dx=-23.93163938537289 evalInputDelta=-3.6835331494344814E-4
Armijo: th(1.2220510246415054E-5)=-10.650470212351518; dx=-23.955657405276497 evalInputDelta=-2.5093073377391306E-5
Armijo: th(8.14700683094337E-7)=-10.65049435851754; dx=-24.012666466021948 evalInputDelta=-9.469073560808283E-7
Armijo: th(5.091879269339606E-8)=-10.650495305424895; dx=-23.96056898751241 evalInputDelta=0.0
Armijo: th(2.995223099611533E-9)=-10.650494831971228; dx=-24.03513556600108 evalInputDelta=-4.734536673822731E-7
Armijo: th(1.6640128331175182E-10)=-10.650494831971228; dx=-23.92440960804641 evalInputDelta=-4.734536673822731E-7
MIN ALPHA (8.757962279565885E-12): th(0.0)=-10.650495305424895
Fitness changed from -10.650495305424895 to -10.650495305424895
Static Iteration Total: 29.0251; Orientation: 0.4818; Line Search: 27.2355
Iteration 34 failed. Error: -10.650495305424895
Previous Error: 0.0 -> -10.650495305424895
Retrying iteration 34
Reset training subject: 5882496134587
Adding measurement 1449cb3b to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-10.650495305424895;dx=-1.2975274515371025E-5
END: th(2.1544346900318838E-10)=-10.650495305424895; dx=-1.1537568969380253E-5 evalInputDelta=0.0
Fitness changed from -10.650495305424895 to -10.650495305424895
Static Iteration Total: 5.3730; Orientation: 0.0196; Line Search: 2.6643
Iteration 35 failed. Error: -10.650495305424895
Previous Error: 0.0 -> -10.650495305424895
Optimization terminated 35
Final threshold in iteration 35: -10.650495305424895 (> -Infinity) after 243.996s (< 3600.000s)
Returns
-10.650495305424895