BasicOptimizer.scala:89 executed in 66.79 seconds (0.885 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 161185407316900
Reset training subject: 161186314481900
Adding measurement 368a49d2 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-2.4201717438868315;dx=-1.8623987502969875E-4
New Minimum: -2.4201717438868315 > -2.4205185498093758
WOLFE (weak): th(2.154434690031884)=-2.4205185498093758; dx=-1.8617000974778105E-4 evalInputDelta=3.468059225442133E-4
New Minimum: -2.4205185498093758 > -2.420866766132593
WOLFE (weak): th(4.308869380063768)=-2.420866766132593; dx=-1.8631695311998402E-4 evalInputDelta=6.950222457615673E-4
New Minimum: -2.420866766132593 > -2.4222575179923007
WOLFE (weak): th(12.926608140191302)=-2.4222575179923007; dx=-1.8639548092054623E-4 evalInputDelta=0.0020857741054691736
New Minimum: -2.4222575179923007 > -2.428528835073237
WOLFE (weak): th(51.70643256076521)=-2.428528835073237; dx=-1.8650719101981524E-4 evalInputDelta=0.008357091186405263
New Minimum: -2.428528835073237 > -2.46197811701226
WOLFE (weak): th(258.53216280382605)=-2.46197811701226; dx=-1.8547879879232725E-4 evalInputDelta=0.041806373125428564
New Minimum: -2.46197811701226 > -2.676635383752516
WOLFE (weak): th(1551.1929768229563)=-2.676635383752516; dx=-1.9558020944369307E-4 evalInputDelta=0.25646363986568455
New Minimum: -2.676635383752516 > -4.040507031371393
END: th(10858.350837760694)=-4.040507031371393; dx=-1.4001825568893503E-4 evalInputDelta=1.6203352874845618
Fitness changed from -2.4201717438868315 to -4.040507031371393
Iteration 1 complete. Error: -4.040507031371393 Total: 9.3796; Orientation: 0.0150; Line Search: 6.8500
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 5a976a4f to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-4.040507031371393;dx=-2.719748077843969E-4
New Minimum: -4.040507031371393 > -6.765619137705288
END: th(23393.607721408407)=-6.765619137705288; dx=-8.452994465931295E-5 evalInputDelta=2.725112106333895
Fitness changed from -4.040507031371393 to -6.765619137705288
Iteration 2 complete. Error: -6.765619137705288 Total: 2.6324; Orientation: 0.0142; Line Search: 1.6971
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 77817700 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-6.765619137705288;dx=-3.206693807668848E-4
New Minimum: -6.765619137705288 > -9.463594052050247
END: th(50400.00000000001)=-9.463594052050247; dx=-1.0582670998832006E-5 evalInputDelta=2.697974914344959
Fitness changed from -6.765619137705288 to -9.463594052050247
Iteration 3 complete. Error: -9.463594052050247 Total: 2.6572; Orientation: 0.0144; Line Search: 1.7178
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 2f0de5b0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.990e+03, gradient 1.843e-02, dot -0.167; [71560dda-5472-497e-9fc6-091190c88078 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -9.463594052050247, -6.765619137705288, -4.040507031371393, -2.4201717438868315
LBFGS Accumulation History: 3 points
Removed measurement 2f0de5b0 to history. Total: 3
th(0)=-9.463594052050247;dx=-3.380143545758251E-4
New Minimum: -9.463594052050247 > -10.31721412357789
WOLF (strong): th(108583.50837760697)=-10.31721412357789; dx=6.764911699544338E-7 evalInputDelta=0.8536200715276419
New Minimum: -10.31721412357789 > -13.312997685600681
END: th(54291.75418880348)=-13.312997685600681; dx=-2.3482421906243283E-5 evalInputDelta=3.849403633550434
Fitness changed from -9.463594052050247 to -13.312997685600681
Iteration 4 complete. Error: -13.312997685600681 Total: 4.0108; Orientation: 0.2771; Line Search: 2.8210
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 2e0f8e29 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.848e+03, gradient 1.679e-02, dot -0.133; [71560dda-5472-497e-9fc6-091190c88078 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -13.312997685600681, -6.765619137705288, -4.040507031371393, -2.4201717438868315
LBFGS Accumulation History: 3 points
Removed measurement 2e0f8e29 to history. Total: 3
th(0)=-13.312997685600681;dx=-2.812881191587086E-4
New Minimum: -13.312997685600681 > -14.392288103711401
WOLF (strong): th(116968.03860704206)=-14.392288103711401; dx=8.024703418742873E-6 evalInputDelta=1.0792904181107197
New Minimum: -14.392288103711401 > -18.05913915730185
END: th(58484.01930352103)=-18.05913915730185; dx=-3.177932767776838E-5 evalInputDelta=4.7461414717011685
Fitness changed from -13.312997685600681 to -18.05913915730185
Iteration 5 complete. Error: -18.05913915730185 Total: 3.7302; Orientation: 0.2739; Line Search: 2.5509
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 1b3d1511 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.175e+03, gradient 1.574e-02, dot -0.135; [71560dda-5472-497e-9fc6-091190c88078 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.05913915730185, -6.765619137705288, -4.040507031371393, -2.4201717438868315
LBFGS Accumulation History: 3 points
Removed measurement 1b3d1511 to history. Total: 3
th(0)=-18.05913915730185;dx=-2.471175036305076E-4
New Minimum: -18.05913915730185 > -18.124143707422668
WOLF (strong): th(126000.00000000004)=-18.124143707422668; dx=2.121831809145797E-5 evalInputDelta=0.0650045501208183
New Minimum: -18.124143707422668 > -22.69062813369161
END: th(63000.00000000002)=-22.69062813369161; dx=-2.776760887761695E-5 evalInputDelta=4.631488976389761
Fitness changed from -18.05913915730185 to -22.69062813369161
Iteration 6 complete. Error: -22.69062813369161 Total: 3.7113; Orientation: 0.2770; Line Search: 2.5154
Adding measurement 2c6f3e4d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.676e+03, gradient 1.547e-02, dot -0.161; [71560dda-5472-497e-9fc6-091190c88078 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -22.69062813369161, -6.765619137705288, -4.040507031371393, -2.4201717438868315
LBFGS Accumulation History: 3 points
Removed measurement 2c6f3e4d to history. Total: 3
th(0)=-22.69062813369161;dx=-2.3817504279185622E-4
Armijo: th(135729.38547200873)=-22.367690060430238; dx=1.4935563136715599E-5 evalInputDelta=-0.3229380732613727
New Minimum: -22.69062813369161 > -26.859955038305753
END: th(67864.69273600436)=-26.859955038305753; dx=-1.8465370775498158E-5 evalInputDelta=4.169326904614142
Fitness changed from -22.69062813369161 to -26.859955038305753
Iteration 7 complete. Error: -26.859955038305753 Total: 3.8180; Orientation: 0.2600; Line Search: 2.5276
Adding measurement 5679df9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.763e+03, gradient 1.486e-02, dot -0.150; [71560dda-5472-497e-9fc6-091190c88078 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -26.859955038305753, -6.765619137705288, -4.040507031371393, -2.4201717438868315
LBFGS Accumulation History: 3 points
Removed measurement 5679df9 to history. Total: 3
th(0)=-26.859955038305753;dx=-2.204494541292321E-4
Armijo: th(146210.0482588026)=-25.903547446997237; dx=3.227256595342712E-5 evalInputDelta=-0.9564075913085155
New Minimum: -26.859955038305753 > -31.30059859340659
END: th(73105.0241294013)=-31.30059859340659; dx=-1.3944526383410656E-5 evalInputDelta=4.440643555100838
Fitness changed from -26.859955038305753 to -31.30059859340659
Iteration 8 complete. Error: -31.30059859340659 Total: 3.8249; Orientation: 0.2708; Line Search: 2.7458
Adding measurement 70b4b54b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.740e+03, gradient 1.456e-02, dot -0.187; [71560dda-5472-497e-9fc6-091190c88078 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -31.30059859340659, -6.765619137705288, -4.040507031371393, -2.4201717438868315
LBFGS Accumulation History: 3 points
Removed measurement 70b4b54b to history. Total: 3
th(0)=-31.30059859340659;dx=-2.1099382291548232E-4
Armijo: th(157500.00000000006)=-30.191609480462372; dx=2.384206272974353E-5 evalInputDelta=-1.1089891129442186
New Minimum: -31.30059859340659 > -35.640908274493924
END: th(78750.00000000003)=-35.640908274493924; dx=-1.6030401222613958E-5 evalInputDelta=4.340309681087334
Fitness changed from -31.30059859340659 to -35.640908274493924
Iteration 9 complete. Error: -35.640908274493924 Total: 3.7691; Orientation: 0.2922; Line Search: 2.6691
Adding measurement 75930c8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.414e+04, gradient 1.390e-02, dot -0.222; [71560dda-5472-497e-9fc6-091190c88078 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -35.640908274493924, -6.765619137705288, -4.040507031371393, -2.4201717438868315
LBFGS Accumulation History: 3 points
Removed measurement 75930c8 to history. Total: 3
th(0)=-35.640908274493924;dx=-1.9281254100243057E-4
Armijo: th(169661.7318400109)=-33.5010497781557; dx=3.171880813423011E-5 evalInputDelta=-2.1398584963382206
New Minimum: -35.640908274493924 > -40.25868844828928
END: th(84830.86592000545)=-40.25868844828928; dx=-1.6151742084711783E-5 evalInputDelta=4.617780173795353
Fitness changed from -35.640908274493924 to -40.25868844828928
Iteration 10 complete. Error: -40.25868844828928 Total: 3.6305; Orientation: 0.2793; Line Search: 2.5458
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 5ed9ed5e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.904e+04, gradient 1.393e-02, dot -0.238; [71560dda-5472-497e-9fc6-091190c88078 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -40.25868844828928, -6.765619137705288, -4.040507031371393, -2.4201717438868315
LBFGS Accumulation History: 3 points
Removed measurement 5ed9ed5e to history. Total: 3
th(0)=-40.25868844828928;dx=-1.936579394266455E-4
Armijo: th(182762.56032350325)=-37.02523060371942; dx=2.6967209362328433E-5 evalInputDelta=-3.2334578445698554
New Minimum: -40.25868844828928 > -44.244393192205
END: th(91381.28016175162)=-44.244393192205; dx=-6.978461652830951E-6 evalInputDelta=3.9857047439157256
Fitness changed from -40.25868844828928 to -44.244393192205
Iteration 11 complete. Error: -44.244393192205 Total: 3.6653; Orientation: 0.2721; Line Search: 2.5054
Adding measurement 5ed6134 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.211e+04, gradient 1.417e-02, dot -0.234; [71560dda-5472-497e-9fc6-091190c88078 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -44.244393192205, -6.765619137705288, -4.040507031371393, -2.4201717438868315
LBFGS Accumulation History: 3 points
Removed measurement 5ed6134 to history. Total: 3
th(0)=-44.244393192205;dx=-2.0014629082228786E-4
Armijo: th(196875.0000000001)=-38.89020639772054; dx=4.6146920067609274E-5 evalInputDelta=-5.3541867944844626
New Minimum: -44.244393192205 > -48.64404993933278
WOLF (strong): th(98437.50000000004)=-48.64404993933278; dx=1.1710612506659976E-5 evalInputDelta=4.3996567471277785
END: th(32812.500000000015)=-48.57864457759406; dx=-1.1051242998190836E-4 evalInputDelta=4.334251385389059
Fitness changed from -44.244393192205 to -48.64404993933278
Iteration 12 complete. Error: -48.64404993933278 Total: 4.5980; Orientation: 0.2567; Line Search: 3.3895
Adding measurement 663a1a63 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.034e+04, gradient 1.564e-02, dot -0.223; [71560dda-5472-497e-9fc6-091190c88078 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -48.64404993933278, -6.765619137705288, -4.040507031371393, -2.4201717438868315
LBFGS Accumulation History: 3 points
Removed measurement 663a1a63 to history. Total: 3
th(0)=-48.64404993933278;dx=-2.4316301855203113E-4
New Minimum: -48.64404993933278 > -53.95673490037797
END: th(70692.38826667122)=-53.95673490037797; dx=-1.3504926221591898E-5 evalInputDelta=5.312684961045186
Fitness changed from -48.64404993933278 to -53.95673490037797
Iteration 13 complete. Error: -53.95673490037797 Total: 2.7748; Orientation: 0.2711; Line Search: 1.6845
Adding measurement 31f9ab5d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.739e+05, gradient 1.249e-02, dot -0.310; [71560dda-5472-497e-9fc6-091190c88078 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -53.95673490037797, -6.765619137705288, -4.040507031371393, -2.4201717438868315
LBFGS Accumulation History: 3 points
Removed measurement 31f9ab5d to history. Total: 3
th(0)=-53.95673490037797;dx=-1.559451752232123E-4
New Minimum: -53.95673490037797 > -55.64717663199336
WOLF (strong): th(152302.13360291938)=-55.64717663199336; dx=1.2863813545583043E-5 evalInputDelta=1.690441731615394
New Minimum: -55.64717663199336 > -58.873008504917436
END: th(76151.06680145969)=-58.873008504917436; dx=-3.8220150901497105E-5 evalInputDelta=4.9162736045394695
Fitness changed from -53.95673490037797 to -58.873008504917436
Iteration 14 complete. Error: -58.873008504917436 Total: 3.7769; Orientation: 0.2736; Line Search: 2.7029
Adding measurement 6472a032 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 4.539e+04, gradient 1.153e-02, dot 0.368; [71560dda-5472-497e-9fc6-091190c88078 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-58.873008504917436;dx=-193.4724672022868
Armijo: th(164062.5000000001)=-9.646194578814892; dx=7.238930754073706E-5 evalInputDelta=-49.226813926102544
Armijo: th(82031.25000000004)=-9.646985111827648; dx=1.4628087419833315E-4 evalInputDelta=-49.22602339308979
Armijo: th(27343.750000000015)=-9.64755621773765; dx=4.3943411933752044E-4 evalInputDelta=-49.225452287179785
Armijo: th(6835.937500000004)=-9.64613378963624; dx=0.0017465640881349546 evalInputDelta=-49.2268747152812
Armijo: th(1367.1875000000007)=-9.677380141890998; dx=0.008871819444351885 evalInputDelta=-49.19562836302644
Armijo: th(227.86458333333346)=-9.819004582425679; dx=0.0520329319398124 evalInputDelta=-49.05400392249176
Armijo: th(32.55208333333335)=-10.252811682455548; dx=0.36318881252276736 evalInputDelta=-48.62019682246189
Armijo: th(4.069010416666669)=-14.977033869579508; dx=2.957714232809579 evalInputDelta=-43.89597463533793
Armijo: th(0.45211226851851877)=-43.784811088440314; dx=3.6564459959940994 evalInputDelta=-15.088197416477122
New Minimum: -58.873008504917436 > -63.35921959521507
END: th(0.045211226851851874)=-63.35921959521507; dx=-67.58182078586466 evalInputDelta=4.486211090297637
Fitness changed from -58.873008504917436 to -63.35921959521507
Iteration 15 complete. Error: -63.35921959521507 Total: 10.7116; Orientation: 0.2613; Line Search: 9.6437
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -63.35921959521507 (> -Infinity) after 66.788s (< 5400.000s)
Returns
-63.35921959521507