BasicOptimizer.scala:89 executed in 68.37 seconds (1.049 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 140571874290000
Reset training subject: 140572780050300
Adding measurement 4f4937e4 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-103.8844569747333;dx=-0.00866989468733339
New Minimum: -103.8844569747333 > -103.90463674465906
WOLFE (weak): th(2.154434690031884)=-103.90463674465906; dx=-0.008147302883494488 evalInputDelta=0.020179769925761093
New Minimum: -103.90463674465906 > -103.92483784506953
WOLFE (weak): th(4.308869380063768)=-103.92483784506953; dx=-0.00812747191231198 evalInputDelta=0.04038087033623583
New Minimum: -103.92483784506953 > -104.00565346855903
WOLFE (weak): th(12.926608140191302)=-104.00565346855903; dx=-0.00814635507724688 evalInputDelta=0.12119649382573527
New Minimum: -104.00565346855903 > -104.36998243570301
WOLFE (weak): th(51.70643256076521)=-104.36998243570301; dx=-0.008171963576589574 evalInputDelta=0.4855254609697113
New Minimum: -104.36998243570301 > -106.32200472088473
WOLFE (weak): th(258.53216280382605)=-106.32200472088473; dx=-0.00819018032429358 evalInputDelta=2.437547746151438
New Minimum: -106.32200472088473 > -118.72840087989145
WOLFE (weak): th(1551.1929768229563)=-118.72840087989145; dx=-0.008386279929147113 evalInputDelta=14.843943905158156
New Minimum: -118.72840087989145 > -202.52204613947967
END: th(10858.350837760694)=-202.52204613947967; dx=-0.006491779686406063 evalInputDelta=98.63758916474637
Fitness changed from -103.8844569747333 to -202.52204613947967
Iteration 1 complete. Error: -202.52204613947967 Total: 9.7066; Orientation: 0.0145; Line Search: 7.1089
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 20df3368 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-202.52204613947967;dx=-0.01120870405990768
New Minimum: -202.52204613947967 > -431.4618536521717
END: th(23393.607721408407)=-431.4618536521717; dx=-0.004900416108424597 evalInputDelta=228.93980751269206
Fitness changed from -202.52204613947967 to -431.4618536521717
Iteration 2 complete. Error: -431.4618536521717 Total: 2.7442; Orientation: 0.0145; Line Search: 1.7668
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 21b4c457 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-431.4618536521717;dx=-0.013998349178393748
New Minimum: -431.4618536521717 > -764.4076173919377
END: th(50400.00000000001)=-764.4076173919377; dx=-0.001935633410501646 evalInputDelta=332.945763739766
Fitness changed from -431.4618536521717 to -764.4076173919377
Iteration 3 complete. Error: -764.4076173919377 Total: 2.6419; Orientation: 0.0156; Line Search: 1.7316
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 393477a1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.852e+04, gradient 1.234e-01, dot -0.207; [0ef002f2-8c87-4463-b8fc-6cac0c6a12e4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -764.4076173919377, -431.4618536521717, -202.52204613947967, -103.8844569747333
LBFGS Accumulation History: 3 points
Removed measurement 393477a1 to history. Total: 3
th(0)=-764.4076173919377;dx=-0.015186435692801679
New Minimum: -764.4076173919377 > -951.6543332688171
WOLF (strong): th(108583.50837760697)=-951.6543332688171; dx=0.001054554451591323 evalInputDelta=187.24671587687942
New Minimum: -951.6543332688171 > -1095.6014970155152
END: th(54291.75418880348)=-1095.6014970155152; dx=-0.0010387249949097802 evalInputDelta=331.19387962357746
Fitness changed from -764.4076173919377 to -1095.6014970155152
Iteration 4 complete. Error: -1095.6014970155152 Total: 3.9932; Orientation: 0.2801; Line Search: 2.7909
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 12003a8c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.646e+04, gradient 1.211e-01, dot -0.170; [0ef002f2-8c87-4463-b8fc-6cac0c6a12e4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1095.6014970155152, -431.4618536521717, -202.52204613947967, -103.8844569747333
LBFGS Accumulation History: 3 points
Removed measurement 12003a8c to history. Total: 3
th(0)=-1095.6014970155152;dx=-0.01462849432391615
New Minimum: -1095.6014970155152 > -1217.3947880302562
WOLF (strong): th(116968.03860704206)=-1217.3947880302562; dx=0.0016057857158407142 evalInputDelta=121.79329101474104
New Minimum: -1217.3947880302562 > -1419.373738114648
END: th(58484.01930352103)=-1419.373738114648; dx=-9.042938608855309E-4 evalInputDelta=323.77224109913277
Fitness changed from -1095.6014970155152 to -1419.373738114648
Iteration 5 complete. Error: -1419.373738114648 Total: 3.8106; Orientation: 0.2891; Line Search: 2.5846
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 676121b6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.765e+04, gradient 1.132e-01, dot -0.164; [0ef002f2-8c87-4463-b8fc-6cac0c6a12e4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1419.373738114648, -431.4618536521717, -202.52204613947967, -103.8844569747333
LBFGS Accumulation History: 3 points
Removed measurement 676121b6 to history. Total: 3
th(0)=-1419.373738114648;dx=-0.012733385013775984
New Minimum: -1419.373738114648 > -1493.3292587840128
WOLF (strong): th(126000.00000000004)=-1493.3292587840128; dx=0.0018988859418856145 evalInputDelta=73.95552066936489
New Minimum: -1493.3292587840128 > -1721.2216439830397
END: th(63000.00000000002)=-1721.2216439830397; dx=-8.075667686010678E-4 evalInputDelta=301.84790586839176
Fitness changed from -1419.373738114648 to -1721.2216439830397
Iteration 6 complete. Error: -1721.2216439830397 Total: 3.6890; Orientation: 0.2782; Line Search: 2.5201
Adding measurement 3b5dca75 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.090e+05, gradient 1.112e-01, dot -0.189; [0ef002f2-8c87-4463-b8fc-6cac0c6a12e4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1721.2216439830397, -431.4618536521717, -202.52204613947967, -103.8844569747333
LBFGS Accumulation History: 3 points
Removed measurement 3b5dca75 to history. Total: 3
th(0)=-1721.2216439830397;dx=-0.012339163777226218
Armijo: th(135729.38547200873)=-1653.4779355004916; dx=0.0023384420647101353 evalInputDelta=-67.74370848254807
New Minimum: -1721.2216439830397 > -1952.7565489645742
END: th(67864.69273600436)=-1952.7565489645742; dx=-1.3050180286466463E-4 evalInputDelta=231.5349049815345
Fitness changed from -1721.2216439830397 to -1952.7565489645742
Iteration 7 complete. Error: -1952.7565489645742 Total: 3.9075; Orientation: 0.2826; Line Search: 2.8331
Adding measurement 1003b4cf to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.540e+05, gradient 1.138e-01, dot -0.192; [0ef002f2-8c87-4463-b8fc-6cac0c6a12e4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1952.7565489645742, -431.4618536521717, -202.52204613947967, -103.8844569747333
LBFGS Accumulation History: 3 points
Removed measurement 1003b4cf to history. Total: 3
th(0)=-1952.7565489645742;dx=-0.012924038651831494
Armijo: th(146210.0482588026)=-1806.3272035756459; dx=0.002677189871317097 evalInputDelta=-146.42934538892837
New Minimum: -1952.7565489645742 > -2176.749013736048
WOLF (strong): th(73105.0241294013)=-2176.749013736048; dx=5.478814971493651E-4 evalInputDelta=223.99246477147358
END: th(24368.3413764671)=-2156.9047583543925; dx=-0.004554147088160418 evalInputDelta=204.14820938981825
Fitness changed from -1952.7565489645742 to -2176.749013736048
Iteration 8 complete. Error: -2176.749013736048 Total: 4.4537; Orientation: 0.2818; Line Search: 3.3684
Adding measurement 470498c1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.527e+05, gradient 1.161e-01, dot -0.203; [0ef002f2-8c87-4463-b8fc-6cac0c6a12e4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2176.749013736048, -431.4618536521717, -202.52204613947967, -103.8844569747333
LBFGS Accumulation History: 3 points
Removed measurement 470498c1 to history. Total: 3
th(0)=-2176.749013736048;dx=-0.013425926236036431
New Minimum: -2176.749013736048 > -2440.273946307172
END: th(52500.00000000002)=-2440.273946307172; dx=-9.263080497836417E-4 evalInputDelta=263.5249325711243
Fitness changed from -2176.749013736048 to -2440.273946307172
Iteration 9 complete. Error: -2440.273946307172 Total: 2.8272; Orientation: 0.2778; Line Search: 1.7334
Adding measurement 3c507538 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.589e+05, gradient 1.023e-01, dot -0.232; [0ef002f2-8c87-4463-b8fc-6cac0c6a12e4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2440.273946307172, -431.4618536521717, -202.52204613947967, -103.8844569747333
LBFGS Accumulation History: 3 points
Removed measurement 3c507538 to history. Total: 3
th(0)=-2440.273946307172;dx=-0.010434012743525657
New Minimum: -2440.273946307172 > -2477.9174117335924
WOLF (strong): th(113107.82122667394)=-2477.9174117335924; dx=0.0013308832146073197 evalInputDelta=37.643465426420335
New Minimum: -2477.9174117335924 > -2624.2354842441773
END: th(56553.91061333697)=-2624.2354842441773; dx=-0.0013219901138057774 evalInputDelta=183.96153793700523
Fitness changed from -2440.273946307172 to -2624.2354842441773
Iteration 10 complete. Error: -2624.2354842441773 Total: 4.0523; Orientation: 0.2763; Line Search: 2.9653
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 1d79e42f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.835e+07, gradient 1.004e-01, dot -0.272; [0ef002f2-8c87-4463-b8fc-6cac0c6a12e4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2624.2354842441773, -431.4618536521717, -202.52204613947967, -103.8844569747333
LBFGS Accumulation History: 3 points
Removed measurement 1d79e42f to history. Total: 3
th(0)=-2624.2354842441773;dx=-0.010061685166617495
New Minimum: -2624.2354842441773 > -2675.9987756561613
WOLF (strong): th(121841.70688233551)=-2675.9987756561613; dx=6.761604981638178E-4 evalInputDelta=51.76329141198403
New Minimum: -2675.9987756561613 > -2779.331838899316
END: th(60920.853441167754)=-2779.331838899316; dx=-9.725101530943788E-4 evalInputDelta=155.09635465513884
Fitness changed from -2624.2354842441773 to -2779.331838899316
Iteration 11 complete. Error: -2779.331838899316 Total: 3.8029; Orientation: 0.2740; Line Search: 2.6005
Adding measurement 44e1cb0f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.242e+05, gradient 1.035e-01, dot -0.147; [0ef002f2-8c87-4463-b8fc-6cac0c6a12e4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2779.331838899316, -431.4618536521717, -202.52204613947967, -103.8844569747333
LBFGS Accumulation History: 3 points
Removed measurement 44e1cb0f to history. Total: 3
th(0)=-2779.331838899316;dx=-0.010695511414057488
Armijo: th(131250.00000000006)=-2648.4272823815563; dx=0.0019067229106336137 evalInputDelta=-130.90455651775983
New Minimum: -2779.331838899316 > -2901.1955533251976
END: th(65625.00000000003)=-2901.1955533251976; dx=-9.926000684765026E-5 evalInputDelta=121.86371442588143
Fitness changed from -2779.331838899316 to -2901.1955533251976
Iteration 12 complete. Error: -2901.1955533251976 Total: 4.0000; Orientation: 0.3499; Line Search: 2.7686
Adding measurement cc51734 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 3.019e+05, gradient 1.088e-01, dot 0.290; [0ef002f2-8c87-4463-b8fc-6cac0c6a12e4 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-2901.1955533251976;dx=-9507.909193192792
Armijo: th(141384.77653334243)=-417.3594020339936; dx=0.0027089370051339772 evalInputDelta=-2483.836151291204
Armijo: th(70692.38826667122)=-417.3349402157801; dx=0.005377716314642017 evalInputDelta=-2483.8606131094175
Armijo: th(23564.129422223737)=-417.3407749674679; dx=0.01623181923582994 evalInputDelta=-2483.8547783577296
Armijo: th(5891.032355555934)=-417.5364434096411; dx=0.0650667276884474 evalInputDelta=-2483.6591099155567
Armijo: th(1178.2064711111868)=-417.9372664460078; dx=0.32510642352107255 evalInputDelta=-2483.2582868791897
Armijo: th(196.3677451851978)=-419.71730342213885; dx=1.9564916987098222 evalInputDelta=-2481.478249903059
Armijo: th(28.05253502645683)=-429.92173446423715; dx=13.856390426056553 evalInputDelta=-2471.2738188609605
Armijo: th(3.506566878307104)=-490.51323554978865; dx=120.69833671815583 evalInputDelta=-2410.682317775409
Armijo: th(0.38961854203412266)=-985.8831311543238; dx=1666.7567565011284 evalInputDelta=-1915.3124221708738
Armijo: th(0.03896185420341226)=-2766.6182969092633; dx=3985.4183887990707 evalInputDelta=-134.57725641593424
New Minimum: -2901.1955533251976 > -2918.2992829033305
END: th(0.003541986745764751)=-2918.2992829033305; dx=-3275.8303180678454 evalInputDelta=17.10372957813297
Fitness changed from -2901.1955533251976 to -2918.2992829033305
Iteration 13 complete. Error: -2918.2992829033305 Total: 11.8824; Orientation: 0.2618; Line Search: 10.4703
Adding measurement 34188882 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 7.429e+04, gradient 1.085e-01, dot -0.256; [0ef002f2-8c87-4463-b8fc-6cac0c6a12e4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2918.2992829033305, -2901.1955533251976, -431.4618536521717, -202.52204613947967, -103.8844569747333
Rejected: LBFGS Orientation magnitude: 3.417e+05, gradient 1.085e-01, dot -0.331; [0ef002f2-8c87-4463-b8fc-6cac0c6a12e4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2918.2992829033305, -2901.1955533251976, -431.4618536521717, -202.52204613947967
LBFGS Accumulation History: 3 points
Removed measurement 34188882 to history. Total: 4
Removed measurement cc51734 to history. Total: 3
th(0)=-2918.2992829033305;dx=-0.011723273696922432
New Minimum: -2918.2992829033305 > -2918.299436358333
END: th(0.007630979116708723)=-2918.299436358333; dx=-0.0071475651035453145 evalInputDelta=1.5345500241892296E-4
Fitness changed from -2918.2992829033305 to -2918.299436358333
Iteration 14 complete. Error: -2918.299436358333 Total: 3.0924; Orientation: 0.6023; Line Search: 1.6784
Adding measurement 5d84b020 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 2.680e+05, gradient 1.085e-01, dot 0.256; [0ef002f2-8c87-4463-b8fc-6cac0c6a12e4 = 1.000/1.000e+00]
th(0)=-2918.299436358333;dx=-7436.998819898097
Armijo: th(0.016440446127946137)=-2913.742254511778; dx=510.46380745244863 evalInputDelta=-4.557181846555068
New Minimum: -2918.299436358333 > -2929.146958854444
END: th(0.008220223063973069)=-2929.146958854444; dx=-1054.446870059447 evalInputDelta=10.847522496111196
Fitness changed from -2918.299436358333 to -2929.146958854444
Iteration 15 complete. Error: -2929.146958854444 Total: 3.6523; Orientation: 0.2775; Line Search: 2.5865
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -2929.146958854444 (> -Infinity) after 68.365s (< 5400.000s)
Returns
-2929.146958854444