BasicOptimizer.scala:89 executed in 44.29 seconds (0.809 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 213609204623100
Reset training subject: 213609831280200
Adding measurement 7260944e to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-10.623193664834183;dx=-1.6598565066201174E-4
New Minimum: -10.623193664834183 > -10.623465152039127
WOLFE (weak): th(2.154434690031884)=-10.623465152039127; dx=-1.6600742996144104E-4 evalInputDelta=2.714872049445205E-4
New Minimum: -10.623465152039127 > -10.623737112145474
WOLFE (weak): th(4.308869380063768)=-10.623737112145474; dx=-1.6578177750355797E-4 evalInputDelta=5.434473112906346E-4
New Minimum: -10.623737112145474 > -10.624822409829937
WOLFE (weak): th(12.926608140191302)=-10.624822409829937; dx=-1.665552531228172E-4 evalInputDelta=0.001628744995754161
New Minimum: -10.624822409829937 > -10.629709470289892
WOLFE (weak): th(51.70643256076521)=-10.629709470289892; dx=-1.6687924142496301E-4 evalInputDelta=0.006515805455709511
New Minimum: -10.629709470289892 > -10.65588173319793
WOLFE (weak): th(258.53216280382605)=-10.65588173319793; dx=-1.6595763389973603E-4 evalInputDelta=0.032688068363746936
New Minimum: -10.65588173319793 > -10.818395580283784
WOLFE (weak): th(1551.1929768229563)=-10.818395580283784; dx=-1.64558117016283E-4 evalInputDelta=0.19520191544960142
New Minimum: -10.818395580283784 > -11.883123907094863
END: th(10858.350837760694)=-11.883123907094863; dx=-1.4722480892294032E-4 evalInputDelta=1.2599302422606797
Fitness changed from -10.623193664834183 to -11.883123907094863
Iteration 1 complete. Error: -11.883123907094863 Total: 6.8898; Orientation: 0.0179; Line Search: 5.1142
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 69b9bee8 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-11.883123907094863;dx=-1.6541769240414878E-4
New Minimum: -11.883123907094863 > -14.085448366835616
END: th(23393.607721408407)=-14.085448366835616; dx=-1.2148870570692265E-4 evalInputDelta=2.202324459740753
Fitness changed from -11.883123907094863 to -14.085448366835616
Iteration 2 complete. Error: -14.085448366835616 Total: 2.0245; Orientation: 0.0182; Line Search: 1.4440
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 5e11d9e7 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-14.085448366835616;dx=-1.720573018124092E-4
New Minimum: -14.085448366835616 > -15.81991994238976
END: th(50400.00000000001)=-15.81991994238976; dx=-6.996110692068407E-5 evalInputDelta=1.734471575554144
Fitness changed from -14.085448366835616 to -15.81991994238976
Iteration 3 complete. Error: -15.81991994238976 Total: 1.8395; Orientation: 0.0166; Line Search: 1.2148
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 7d679029 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.060e+04, gradient 1.533e-02, dot -0.318; [be5fde30-c55d-461c-a22f-05477e32c4aa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -15.81991994238976, -14.085448366835616, -11.883123907094863, -10.623193664834183
LBFGS Accumulation History: 3 points
Removed measurement 7d679029 to history. Total: 3
th(0)=-15.81991994238976;dx=-2.3409174635877257E-4
Armijo: th(108583.50837760697)=-11.115496298460492; dx=-1.8319497780353516E-5 evalInputDelta=-4.7044236439292675
New Minimum: -15.81991994238976 > -15.82892521309822
END: th(54291.75418880348)=-15.82892521309822; dx=-5.4237220196086655E-5 evalInputDelta=0.009005270708460245
Fitness changed from -15.81991994238976 to -15.82892521309822
Iteration 4 complete. Error: -15.82892521309822 Total: 2.8508; Orientation: 0.3143; Line Search: 1.8856
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 2996da1a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.027e+04, gradient 1.539e-02, dot -0.265; [be5fde30-c55d-461c-a22f-05477e32c4aa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -15.82892521309822, -14.085448366835616, -11.883123907094863, -10.623193664834183
LBFGS Accumulation History: 3 points
Removed measurement 2996da1a to history. Total: 3
th(0)=-15.82892521309822;dx=-2.3664959017520348E-4
Armijo: th(116968.03860704206)=-11.256349691108944; dx=-2.197880428645258E-5 evalInputDelta=-4.572575521989275
New Minimum: -15.82892521309822 > -16.78774011914478
END: th(58484.01930352103)=-16.78774011914478; dx=-7.502798157772993E-5 evalInputDelta=0.958814906046559
Fitness changed from -15.82892521309822 to -16.78774011914478
Iteration 5 complete. Error: -16.78774011914478 Total: 2.7124; Orientation: 0.3012; Line Search: 1.8380
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 229b6daf to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.082e+03, gradient 1.559e-02, dot -0.217; [be5fde30-c55d-461c-a22f-05477e32c4aa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.78774011914478, -14.085448366835616, -11.883123907094863, -10.623193664834183
LBFGS Accumulation History: 3 points
Removed measurement 229b6daf to history. Total: 3
th(0)=-16.78774011914478;dx=-2.4329187669871568E-4
Armijo: th(126000.00000000004)=-9.966535017964524; dx=-2.235463299732217E-5 evalInputDelta=-6.821205101180254
Armijo: th(63000.00000000002)=-16.68822290380199; dx=-7.256604002628145E-5 evalInputDelta=-0.09951721534278946
New Minimum: -16.78774011914478 > -18.474513797993218
END: th(21000.000000000007)=-18.474513797993218; dx=-1.6760022561585362E-4 evalInputDelta=1.6867736788484393
Fitness changed from -16.78774011914478 to -18.474513797993218
Iteration 6 complete. Error: -18.474513797993218 Total: 3.3175; Orientation: 0.3008; Line Search: 2.4072
Adding measurement 512f5e5b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.189e+04, gradient 1.417e-02, dot -0.254; [be5fde30-c55d-461c-a22f-05477e32c4aa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.474513797993218, -14.085448366835616, -11.883123907094863, -10.623193664834183
LBFGS Accumulation History: 3 points
Removed measurement 512f5e5b to history. Total: 3
th(0)=-18.474513797993218;dx=-1.994140888220909E-4
New Minimum: -18.474513797993218 > -19.220549341249267
END: th(45243.12849066957)=-19.220549341249267; dx=-1.0095831535324988E-4 evalInputDelta=0.746035543256049
Fitness changed from -18.474513797993218 to -19.220549341249267
Iteration 7 complete. Error: -19.220549341249267 Total: 2.3672; Orientation: 0.2828; Line Search: 1.2388
Adding measurement 27c088da to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.108e+04, gradient 1.493e-02, dot -0.216; [be5fde30-c55d-461c-a22f-05477e32c4aa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.220549341249267, -14.085448366835616, -11.883123907094863, -10.623193664834183
LBFGS Accumulation History: 3 points
Removed measurement 27c088da to history. Total: 3
th(0)=-19.220549341249267;dx=-2.227175398665236E-4
Armijo: th(97473.3655058684)=-15.392218627116339; dx=-3.755560459905757E-5 evalInputDelta=-3.828330714132928
New Minimum: -19.220549341249267 > -20.02104770615972
END: th(48736.6827529342)=-20.02104770615972; dx=-8.59255564909723E-5 evalInputDelta=0.8004983649104531
Fitness changed from -19.220549341249267 to -20.02104770615972
Iteration 8 complete. Error: -20.02104770615972 Total: 2.6471; Orientation: 0.2986; Line Search: 1.7875
Adding measurement 7fbd2dd9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.195e+04, gradient 1.463e-02, dot -0.211; [be5fde30-c55d-461c-a22f-05477e32c4aa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -20.02104770615972, -14.085448366835616, -11.883123907094863, -10.623193664834183
LBFGS Accumulation History: 3 points
Removed measurement 7fbd2dd9 to history. Total: 3
th(0)=-20.02104770615972;dx=-2.1375042106336638E-4
Armijo: th(105000.00000000004)=-15.681339461362551; dx=-3.1641392916887114E-5 evalInputDelta=-4.339708244797169
New Minimum: -20.02104770615972 > -20.49818918381347
END: th(52500.00000000002)=-20.49818918381347; dx=-7.76564561280688E-5 evalInputDelta=0.47714147765374904
Fitness changed from -20.02104770615972 to -20.49818918381347
Iteration 9 complete. Error: -20.49818918381347 Total: 2.7622; Orientation: 0.3246; Line Search: 1.8416
Adding measurement 1cacfb56 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.202e+04, gradient 1.499e-02, dot -0.200; [be5fde30-c55d-461c-a22f-05477e32c4aa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -20.49818918381347, -14.085448366835616, -11.883123907094863, -10.623193664834183
LBFGS Accumulation History: 3 points
Removed measurement 1cacfb56 to history. Total: 3
th(0)=-20.49818918381347;dx=-2.2429083431238625E-4
Armijo: th(113107.82122667394)=-15.508440104037902; dx=-2.6804682613927124E-5 evalInputDelta=-4.989749079775567
New Minimum: -20.49818918381347 > -21.039747010414324
END: th(56553.91061333697)=-21.039747010414324; dx=-7.488409583722811E-5 evalInputDelta=0.5415578266008545
Fitness changed from -20.49818918381347 to -21.039747010414324
Iteration 10 complete. Error: -21.039747010414324 Total: 2.7184; Orientation: 0.2935; Line Search: 1.8398
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 11d1991a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.290e+04, gradient 1.451e-02, dot -0.205; [be5fde30-c55d-461c-a22f-05477e32c4aa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -21.039747010414324, -14.085448366835616, -11.883123907094863, -10.623193664834183
LBFGS Accumulation History: 3 points
Removed measurement 11d1991a to history. Total: 3
th(0)=-21.039747010414324;dx=-2.1045319113054947E-4
Armijo: th(121841.70688233551)=-16.13974453557517; dx=-2.510023336759169E-5 evalInputDelta=-4.900002474839155
New Minimum: -21.039747010414324 > -21.849268125082922
END: th(60920.853441167754)=-21.849268125082922; dx=-7.7551932657284E-5 evalInputDelta=0.8095211146685983
Fitness changed from -21.039747010414324 to -21.849268125082922
Iteration 11 complete. Error: -21.849268125082922 Total: 3.0445; Orientation: 0.4961; Line Search: 1.9670
Adding measurement 1c2931da to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.282e+04, gradient 1.425e-02, dot -0.194; [be5fde30-c55d-461c-a22f-05477e32c4aa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -21.849268125082922, -14.085448366835616, -11.883123907094863, -10.623193664834183
LBFGS Accumulation History: 3 points
Removed measurement 1c2931da to history. Total: 3
th(0)=-21.849268125082922;dx=-2.0276317578393234E-4
Armijo: th(131250.00000000006)=-15.349909655104597; dx=-2.0346273571069862E-5 evalInputDelta=-6.499358469978326
New Minimum: -21.849268125082922 > -22.063930168968902
END: th(65625.00000000003)=-22.063930168968902; dx=-6.661592804207027E-5 evalInputDelta=0.21466204388597987
Fitness changed from -21.849268125082922 to -22.063930168968902
Iteration 12 complete. Error: -22.063930168968902 Total: 2.6936; Orientation: 0.2956; Line Search: 1.8430
Adding measurement 5ada573 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.307e+04, gradient 1.455e-02, dot -0.176; [be5fde30-c55d-461c-a22f-05477e32c4aa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -22.063930168968902, -14.085448366835616, -11.883123907094863, -10.623193664834183
LBFGS Accumulation History: 3 points
Removed measurement 5ada573 to history. Total: 3
th(0)=-22.063930168968902;dx=-2.1108130438003252E-4
Armijo: th(141384.77653334243)=-13.925131047787836; dx=-1.0332147344068301E-5 evalInputDelta=-8.138799121181066
New Minimum: -22.063930168968902 > -22.1745049556042
END: th(70692.38826667122)=-22.1745049556042; dx=-6.313860331144485E-5 evalInputDelta=0.11057478663529707
Fitness changed from -22.063930168968902 to -22.1745049556042
Iteration 13 complete. Error: -22.1745049556042 Total: 2.6891; Orientation: 0.2941; Line Search: 1.8380
Adding measurement b731ef4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.409e+04, gradient 1.519e-02, dot -0.166; [be5fde30-c55d-461c-a22f-05477e32c4aa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -22.1745049556042, -14.085448366835616, -11.883123907094863, -10.623193664834183
LBFGS Accumulation History: 3 points
Removed measurement b731ef4 to history. Total: 3
th(0)=-22.1745049556042;dx=-2.3013212506992463E-4
Armijo: th(152302.13360291938)=-11.491442596821123; dx=-4.507242711703974E-6 evalInputDelta=-10.683062358783076
Armijo: th(76151.06680145969)=-21.585765676640357; dx=-5.506667081004071E-5 evalInputDelta=-0.5887392789638426
New Minimum: -22.1745049556042 > -23.89659190131058
END: th(25383.688933819896)=-23.89659190131058; dx=-1.48133954240357E-4 evalInputDelta=1.7220869457063799
Fitness changed from -22.1745049556042 to -23.89659190131058
Iteration 14 complete. Error: -23.89659190131058 Total: 3.6110; Orientation: 0.2933; Line Search: 2.7527
Adding measurement 6f110125 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.845e+04, gradient 1.321e-02, dot -0.199; [be5fde30-c55d-461c-a22f-05477e32c4aa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -23.89659190131058, -14.085448366835616, -11.883123907094863, -10.623193664834183
LBFGS Accumulation History: 3 points
Removed measurement 6f110125 to history. Total: 3
th(0)=-23.89659190131058;dx=-1.7399930329703716E-4
New Minimum: -23.89659190131058 > -24.78953006335683
END: th(54687.50000000003)=-24.78953006335683; dx=-8.531753279957132E-5 evalInputDelta=0.8929381620462493
Fitness changed from -23.89659190131058 to -24.78953006335683
Iteration 15 complete. Error: -24.78953006335683 Total: 2.1014; Orientation: 0.2814; Line Search: 1.2622
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -24.78953006335683 (> -Infinity) after 44.289s (< 5400.000s)
Returns
-24.78953006335683