BasicOptimizer.scala:89 executed in 10.32 seconds (0.869 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 138967891971400
Reset training subject: 138968217925600
Adding measurement 7e191fda to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.6914668020946683;dx=-4.6162031949545084E-5
New Minimum: -0.6914668020946683 > -0.6916049226007046
WOLFE (weak): th(2.154434690031884)=-0.6916049226007046; dx=-4.728316311497488E-5 evalInputDelta=1.381205060363433E-4
New Minimum: -0.6916049226007046 > -0.6917431087409214
WOLFE (weak): th(4.308869380063768)=-0.6917431087409214; dx=-4.690506236273098E-5 evalInputDelta=2.763066462531105E-4
New Minimum: -0.6917431087409214 > -0.6922944598133989
WOLFE (weak): th(12.926608140191302)=-0.6922944598133989; dx=-4.726074339980709E-5 evalInputDelta=8.276577187306122E-4
New Minimum: -0.6922944598133989 > -0.6947714244251078
WOLFE (weak): th(51.70643256076521)=-0.6947714244251078; dx=-4.7079653857786526E-5 evalInputDelta=0.0033046223304394884
New Minimum: -0.6947714244251078 > -0.7080282866380303
WOLFE (weak): th(258.53216280382605)=-0.7080282866380303; dx=-4.714199166606224E-5 evalInputDelta=0.016561484543362037
New Minimum: -0.7080282866380303 > -0.7912838889137014
WOLFE (weak): th(1551.1929768229563)=-0.7912838889137014; dx=-4.885916611693013E-5 evalInputDelta=0.09981708681903312
New Minimum: -0.7912838889137014 > -1.3497211165778413
END: th(10858.350837760694)=-1.3497211165778413; dx=-3.732234261484763E-5 evalInputDelta=0.6582543144831731
Fitness changed from -0.6914668020946683 to -1.3497211165778413
Iteration 1 complete. Error: -1.3497211165778413 Total: 2.2975; Orientation: 0.0282; Line Search: 1.5970
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 3291d9c2 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-1.3497211165778413;dx=-6.261074143274038E-5
New Minimum: -1.3497211165778413 > -2.6303172132287984
END: th(23393.607721408407)=-2.6303172132287984; dx=-2.929929848984565E-5 evalInputDelta=1.280596096650957
Fitness changed from -1.3497211165778413 to -2.6303172132287984
Iteration 2 complete. Error: -2.6303172132287984 Total: 0.4522; Orientation: 0.0023; Line Search: 0.2863
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 3c82bac3 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-2.6303172132287984;dx=-6.97031803257275E-5
New Minimum: -2.6303172132287984 > -4.336203682220481
END: th(50400.00000000001)=-4.336203682220481; dx=-1.411701810111663E-5 evalInputDelta=1.7058864689916828
Fitness changed from -2.6303172132287984 to -4.336203682220481
Iteration 3 complete. Error: -4.336203682220481 Total: 0.4242; Orientation: 0.0023; Line Search: 0.2619
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 43549c6c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.540e+03, gradient 8.136e-03, dot -0.257; [bbd8927f-eaa1-494f-9c28-99585308f4e0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.336203682220481, -2.6303172132287984, -1.3497211165778413, -0.6914668020946683
LBFGS Accumulation History: 3 points
Removed measurement 43549c6c to history. Total: 3
th(0)=-4.336203682220481;dx=-5.8761750836668585E-5
New Minimum: -4.336203682220481 > -5.233405199428372
END: th(108583.50837760697)=-5.233405199428372; dx=-1.0013807630124866E-6 evalInputDelta=0.8972015172078907
Fitness changed from -4.336203682220481 to -5.233405199428372
Iteration 4 complete. Error: -5.233405199428372 Total: 0.9892; Orientation: 0.0646; Line Search: 0.7833
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 4e13af1b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.911e+03, gradient 7.969e-03, dot -0.245; [bbd8927f-eaa1-494f-9c28-99585308f4e0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.233405199428372, -2.6303172132287984, -1.3497211165778413, -0.6914668020946683
LBFGS Accumulation History: 3 points
Removed measurement 4e13af1b to history. Total: 3
th(0)=-5.233405199428372;dx=-5.8854992191626354E-5
New Minimum: -5.233405199428372 > -6.59233647629274
WOLF (strong): th(233936.07721408413)=-6.59233647629274; dx=3.635330872168647E-6 evalInputDelta=1.358931276864368
New Minimum: -6.59233647629274 > -7.941913430543984
END: th(116968.03860704206)=-7.941913430543984; dx=-3.993678417766843E-6 evalInputDelta=2.708508231115612
Fitness changed from -5.233405199428372 to -7.941913430543984
Iteration 5 complete. Error: -7.941913430543984 Total: 0.5557; Orientation: 0.0386; Line Search: 0.3612
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 57859e2c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.307e+03, gradient 7.159e-03, dot -0.208; [bbd8927f-eaa1-494f-9c28-99585308f4e0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -7.941913430543984, -2.6303172132287984, -1.3497211165778413, -0.6914668020946683
LBFGS Accumulation History: 3 points
Removed measurement 57859e2c to history. Total: 3
th(0)=-7.941913430543984;dx=-4.748408318256753E-5
New Minimum: -7.941913430543984 > -8.940073327433922
WOLF (strong): th(252000.0000000001)=-8.940073327433922; dx=3.7894624015456664E-6 evalInputDelta=0.9981598968899377
New Minimum: -8.940073327433922 > -10.433033295622133
END: th(126000.00000000004)=-10.433033295622133; dx=-5.738668910131375E-6 evalInputDelta=2.491119865078149
Fitness changed from -7.941913430543984 to -10.433033295622133
Iteration 6 complete. Error: -10.433033295622133 Total: 0.5307; Orientation: 0.0363; Line Search: 0.3596
Adding measurement 7a8e35d1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.319e+03, gradient 6.411e-03, dot -0.271; [bbd8927f-eaa1-494f-9c28-99585308f4e0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.433033295622133, -2.6303172132287984, -1.3497211165778413, -0.6914668020946683
LBFGS Accumulation History: 3 points
Removed measurement 7a8e35d1 to history. Total: 3
th(0)=-10.433033295622133;dx=-3.709491061907131E-5
New Minimum: -10.433033295622133 > -11.153606836414324
WOLF (strong): th(271458.77094401745)=-11.153606836414324; dx=4.046661041914368E-6 evalInputDelta=0.7205735407921914
New Minimum: -11.153606836414324 > -12.547660531792728
END: th(135729.38547200873)=-12.547660531792728; dx=-3.627787323448361E-6 evalInputDelta=2.1146272361705947
Fitness changed from -10.433033295622133 to -12.547660531792728
Iteration 7 complete. Error: -12.547660531792728 Total: 0.5017; Orientation: 0.0296; Line Search: 0.3630
Adding measurement 73e0c775 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.634e+03, gradient 6.193e-03, dot -0.291; [bbd8927f-eaa1-494f-9c28-99585308f4e0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.547660531792728, -2.6303172132287984, -1.3497211165778413, -0.6914668020946683
LBFGS Accumulation History: 3 points
Removed measurement 73e0c775 to history. Total: 3
th(0)=-12.547660531792728;dx=-3.480222535627841E-5
New Minimum: -12.547660531792728 > -12.757936234059624
WOLF (strong): th(292420.0965176052)=-12.757936234059624; dx=2.2998447705827565E-6 evalInputDelta=0.2102757022668964
New Minimum: -12.757936234059624 > -14.327039316797245
END: th(146210.0482588026)=-14.327039316797245; dx=-2.5014274747893003E-6 evalInputDelta=1.7793787850045177
Fitness changed from -12.547660531792728 to -14.327039316797245
Iteration 8 complete. Error: -14.327039316797245 Total: 0.4953; Orientation: 0.0288; Line Search: 0.3536
Adding measurement 1d92d132 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.376e+03, gradient 6.138e-03, dot -0.290; [bbd8927f-eaa1-494f-9c28-99585308f4e0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -14.327039316797245, -2.6303172132287984, -1.3497211165778413, -0.6914668020946683
LBFGS Accumulation History: 3 points
Removed measurement 1d92d132 to history. Total: 3
th(0)=-14.327039316797245;dx=-3.466818308920019E-5
Armijo: th(315000.0000000001)=-13.975114662465241; dx=3.9549040333347726E-6 evalInputDelta=-0.35192465433200404
New Minimum: -14.327039316797245 > -16.070148640267796
WOLF (strong): th(157500.00000000006)=-16.070148640267796; dx=8.385967842503994E-7 evalInputDelta=1.7431093234705504
END: th(52500.00000000002)=-15.773696471843554; dx=-1.619545840489556E-5 evalInputDelta=1.4466571550463083
Fitness changed from -14.327039316797245 to -16.070148640267796
Iteration 9 complete. Error: -16.070148640267796 Total: 0.6127; Orientation: 0.0281; Line Search: 0.4729
Adding measurement 1cf45f46 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.212e+04, gradient 6.992e-03, dot -0.283; [bbd8927f-eaa1-494f-9c28-99585308f4e0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.070148640267796, -2.6303172132287984, -1.3497211165778413, -0.6914668020946683
LBFGS Accumulation History: 3 points
Removed measurement 1cf45f46 to history. Total: 3
th(0)=-16.070148640267796;dx=-4.5503916089991145E-5
New Minimum: -16.070148640267796 > -17.649818381059195
END: th(113107.82122667394)=-17.649818381059195; dx=-4.2834918684959205E-7 evalInputDelta=1.5796697407913989
Fitness changed from -16.070148640267796 to -17.649818381059195
Iteration 10 complete. Error: -17.649818381059195 Total: 0.4915; Orientation: 0.0366; Line Search: 0.3115
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 5a4929f5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.378e+04, gradient 5.885e-03, dot -0.326; [bbd8927f-eaa1-494f-9c28-99585308f4e0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.649818381059195, -2.6303172132287984, -1.3497211165778413, -0.6914668020946683
LBFGS Accumulation History: 3 points
Removed measurement 5a4929f5 to history. Total: 3
th(0)=-17.649818381059195;dx=-3.19074775688367E-5
New Minimum: -17.649818381059195 > -18.353307943805056
WOLF (strong): th(243683.41376467102)=-18.353307943805056; dx=2.5496271985211102E-6 evalInputDelta=0.7034895627458617
New Minimum: -18.353307943805056 > -19.39848788036215
END: th(121841.70688233551)=-19.39848788036215; dx=-6.420048563978274E-6 evalInputDelta=1.748669499302956
Fitness changed from -17.649818381059195 to -19.39848788036215
Iteration 11 complete. Error: -19.39848788036215 Total: 0.5688; Orientation: 0.0373; Line Search: 0.3699
Adding measurement 16f108a4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.760e+04, gradient 5.306e-03, dot -0.362; [bbd8927f-eaa1-494f-9c28-99585308f4e0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.39848788036215, -2.6303172132287984, -1.3497211165778413, -0.6914668020946683
LBFGS Accumulation History: 3 points
Removed measurement 16f108a4 to history. Total: 3
th(0)=-19.39848788036215;dx=-2.585100769102057E-5
Armijo: th(262500.0000000001)=-19.321625127181917; dx=1.5331953318731564E-6 evalInputDelta=-0.07686275318023306
New Minimum: -19.39848788036215 > -20.304645104075682
END: th(131250.00000000006)=-20.304645104075682; dx=-5.3248311565004755E-6 evalInputDelta=0.9061572237135316
Fitness changed from -19.39848788036215 to -20.304645104075682
Iteration 12 complete. Error: -20.304645104075682 Total: 0.5109; Orientation: 0.0331; Line Search: 0.3630
Adding measurement 2f738b8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.715e+04, gradient 5.519e-03, dot -0.374; [bbd8927f-eaa1-494f-9c28-99585308f4e0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -20.304645104075682, -2.6303172132287984, -1.3497211165778413, -0.6914668020946683
LBFGS Accumulation History: 3 points
Removed measurement 2f738b8 to history. Total: 3
th(0)=-20.304645104075682;dx=-2.7316956626202973E-5
Armijo: th(282769.55306668486)=-19.264637087814624; dx=4.3192145176374526E-6 evalInputDelta=-1.0400080162610585
New Minimum: -20.304645104075682 > -21.017515801392516
END: th(141384.77653334243)=-21.017515801392516; dx=-6.545720198994036E-7 evalInputDelta=0.7128706973168342
Fitness changed from -20.304645104075682 to -21.017515801392516
Iteration 13 complete. Error: -21.017515801392516 Total: 0.9039; Orientation: 0.0275; Line Search: 0.7662
Adding measurement 5b0eb43f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.667e+04, gradient 6.099e-03, dot -0.361; [bbd8927f-eaa1-494f-9c28-99585308f4e0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -21.017515801392516, -2.6303172132287984, -1.3497211165778413, -0.6914668020946683
LBFGS Accumulation History: 3 points
Removed measurement 5b0eb43f to history. Total: 3
th(0)=-21.017515801392516;dx=-3.465866361581813E-5
Armijo: th(304604.26720583875)=-18.799820999319426; dx=8.107033771605601E-6 evalInputDelta=-2.2176948020730904
New Minimum: -21.017515801392516 > -21.724277399423848
WOLF (strong): th(152302.13360291938)=-21.724277399423848; dx=4.46652600348177E-6 evalInputDelta=0.7067615980313313
New Minimum: -21.724277399423848 > -22.222877118067853
END: th(50767.37786763979)=-22.222877118067853; dx=-1.5686764692266687E-5 evalInputDelta=1.2053613166753365
Fitness changed from -21.017515801392516 to -22.222877118067853
Iteration 14 complete. Error: -22.222877118067853 Total: 0.5883; Orientation: 0.0286; Line Search: 0.4519
Adding measurement 494e6dbb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.717e+04, gradient 4.819e-03, dot -0.452; [bbd8927f-eaa1-494f-9c28-99585308f4e0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -22.222877118067853, -2.6303172132287984, -1.3497211165778413, -0.6914668020946683
LBFGS Accumulation History: 3 points
Removed measurement 494e6dbb to history. Total: 3
th(0)=-22.222877118067853;dx=-2.129581051534199E-5
New Minimum: -22.222877118067853 > -22.975745432852218
END: th(109375.00000000006)=-22.975745432852218; dx=-6.92267226329029E-6 evalInputDelta=0.7528683147843651
Fitness changed from -22.222877118067853 to -22.975745432852218
Iteration 15 complete. Error: -22.975745432852218 Total: 0.3697; Orientation: 0.0270; Line Search: 0.2333
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -22.975745432852218 (> -Infinity) after 10.304s (< 5400.000s)
Returns
-22.975745432852218