BasicOptimizer.scala:89 executed in 9.15 seconds (0.689 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 138234877863700
Reset training subject: 138235024314600
Adding measurement 2c781dc1 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-2.190968227098594;dx=-8.328364234793492E-4
New Minimum: -2.190968227098594 > -2.193017495140371
WOLFE (weak): th(2.154434690031884)=-2.193017495140371; dx=-8.545933021742254E-4 evalInputDelta=0.002049268041777186
New Minimum: -2.193017495140371 > -2.1950830242033676
WOLFE (weak): th(4.308869380063768)=-2.1950830242033676; dx=-8.500517994300002E-4 evalInputDelta=0.004114797104773693
New Minimum: -2.1950830242033676 > -2.2033794809475675
WOLFE (weak): th(12.926608140191302)=-2.2033794809475675; dx=-8.732965174378879E-4 evalInputDelta=0.012411253848973569
New Minimum: -2.2033794809475675 > -2.241961409978309
WOLFE (weak): th(51.70643256076521)=-2.241961409978309; dx=-8.858267588252299E-4 evalInputDelta=0.05099318287971499
New Minimum: -2.241961409978309 > -2.4910251328176747
WOLFE (weak): th(258.53216280382605)=-2.4910251328176747; dx=-0.0011440438827526442 evalInputDelta=0.3000569057190807
New Minimum: -2.4910251328176747 > -5.1251997761125665
WOLFE (weak): th(1551.1929768229563)=-5.1251997761125665; dx=-0.0015123607228048983 evalInputDelta=2.9342315490139725
New Minimum: -5.1251997761125665 > -17.942207758864807
END: th(10858.350837760694)=-17.942207758864807; dx=-3.330561088163699E-4 evalInputDelta=15.751239531766213
Fitness changed from -2.190968227098594 to -17.942207758864807
Iteration 1 complete. Error: -17.942207758864807 Total: 1.3187; Orientation: 0.0015; Line Search: 0.9368
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 495204f2 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-17.942207758864807;dx=-0.010503306676699738
New Minimum: -17.942207758864807 > -42.31079252979662
END: th(23393.607721408407)=-42.31079252979662; dx=-8.693528357923778E-5 evalInputDelta=24.368584770931815
Fitness changed from -17.942207758864807 to -42.31079252979662
Iteration 2 complete. Error: -42.31079252979662 Total: 0.3620; Orientation: 0.0015; Line Search: 0.2356
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 5d45f3c9 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-42.31079252979662;dx=-0.003950169113853271
New Minimum: -42.31079252979662 > -81.2803736693132
END: th(50400.00000000001)=-81.2803736693132; dx=-6.129982722049342E-5 evalInputDelta=38.969581139516585
Fitness changed from -42.31079252979662 to -81.2803736693132
Iteration 3 complete. Error: -81.2803736693132 Total: 0.3556; Orientation: 0.0020; Line Search: 0.2265
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 6c8b1626 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.848e+03, gradient 5.892e-02, dot -0.061; [5b66a2e5-0553-4367-adfb-fa22b5b08dda = 1.000/1.000e+00]
Orientation rejected. Popping history element from -81.2803736693132, -42.31079252979662, -17.942207758864807, -2.190968227098594
LBFGS Accumulation History: 3 points
Removed measurement 6c8b1626 to history. Total: 3
th(0)=-81.2803736693132;dx=-0.003250974392572232
New Minimum: -81.2803736693132 > -104.08562739071041
WOLF (strong): th(108583.50837760697)=-104.08562739071041; dx=1.1178053204438976E-4 evalInputDelta=22.805253721397207
New Minimum: -104.08562739071041 > -139.76698132578406
END: th(54291.75418880348)=-139.76698132578406; dx=-9.824793183894823E-5 evalInputDelta=58.48660765647085
Fitness changed from -81.2803736693132 to -139.76698132578406
Iteration 4 complete. Error: -139.76698132578406 Total: 0.5099; Orientation: 0.0279; Line Search: 0.3523
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 79811522 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.358e+03, gradient 5.913e-02, dot -0.080; [5b66a2e5-0553-4367-adfb-fa22b5b08dda = 1.000/1.000e+00]
Orientation rejected. Popping history element from -139.76698132578406, -42.31079252979662, -17.942207758864807, -2.190968227098594
LBFGS Accumulation History: 3 points
Removed measurement 79811522 to history. Total: 3
th(0)=-139.76698132578406;dx=-0.003335351464597913
Armijo: th(116968.03860704206)=-117.29831861813994; dx=1.6239886352236878E-4 evalInputDelta=-22.468662707644114
New Minimum: -139.76698132578406 > -167.60928162579248
WOLF (strong): th(58484.01930352103)=-167.60928162579248; dx=5.766438696772765E-5 evalInputDelta=27.842300300008418
New Minimum: -167.60928162579248 > -180.2202397241334
END: th(19494.673101173677)=-180.2202397241334; dx=-0.0011179820487866106 evalInputDelta=40.453258398349334
Fitness changed from -139.76698132578406 to -180.2202397241334
Iteration 5 complete. Error: -180.2202397241334 Total: 0.6900; Orientation: 0.0289; Line Search: 0.5364
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 5dadb141 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.932e+03, gradient 5.104e-02, dot -0.106; [5b66a2e5-0553-4367-adfb-fa22b5b08dda = 1.000/1.000e+00]
Orientation rejected. Popping history element from -180.2202397241334, -42.31079252979662, -17.942207758864807, -2.190968227098594
LBFGS Accumulation History: 3 points
Removed measurement 5dadb141 to history. Total: 3
th(0)=-180.2202397241334;dx=-0.0024743280606491567
New Minimum: -180.2202397241334 > -191.0449175486084
END: th(42000.000000000015)=-191.0449175486084; dx=-5.76270701542107E-4 evalInputDelta=10.824677824475003
Fitness changed from -180.2202397241334 to -191.0449175486084
Iteration 6 complete. Error: -191.0449175486084 Total: 0.4082; Orientation: 0.0289; Line Search: 0.2488
Adding measurement 231da4a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.285e+03, gradient 6.065e-02, dot -0.122; [5b66a2e5-0553-4367-adfb-fa22b5b08dda = 1.000/1.000e+00]
Orientation rejected. Popping history element from -191.0449175486084, -42.31079252979662, -17.942207758864807, -2.190968227098594
LBFGS Accumulation History: 3 points
Removed measurement 231da4a to history. Total: 3
th(0)=-191.0449175486084;dx=-0.003428933861969901
Armijo: th(90486.25698133915)=-139.20286852030847; dx=3.991315663708705E-4 evalInputDelta=-51.842049028299925
New Minimum: -191.0449175486084 > -199.5190600820971
END: th(45243.12849066957)=-199.5190600820971; dx=-8.945142213641539E-5 evalInputDelta=8.474142533488703
Fitness changed from -191.0449175486084 to -199.5190600820971
Iteration 7 complete. Error: -199.5190600820971 Total: 0.5121; Orientation: 0.0278; Line Search: 0.3614
Adding measurement cee9887 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.161e+04, gradient 5.880e-02, dot -0.137; [5b66a2e5-0553-4367-adfb-fa22b5b08dda = 1.000/1.000e+00]
Orientation rejected. Popping history element from -199.5190600820971, -42.31079252979662, -17.942207758864807, -2.190968227098594
LBFGS Accumulation History: 3 points
Removed measurement cee9887 to history. Total: 3
th(0)=-199.5190600820971;dx=-0.0032037390567620657
Armijo: th(97473.3655058684)=-154.57769494641929; dx=3.1220020346658386E-4 evalInputDelta=-44.94136513567781
New Minimum: -199.5190600820971 > -216.0169503055568
END: th(48736.6827529342)=-216.0169503055568; dx=-1.3909045313139658E-4 evalInputDelta=16.497890223459706
Fitness changed from -199.5190600820971 to -216.0169503055568
Iteration 8 complete. Error: -216.0169503055568 Total: 0.5298; Orientation: 0.0304; Line Search: 0.3749
Adding measurement 54a65417 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.746e+04, gradient 6.019e-02, dot -0.167; [5b66a2e5-0553-4367-adfb-fa22b5b08dda = 1.000/1.000e+00]
Orientation rejected. Popping history element from -216.0169503055568, -42.31079252979662, -17.942207758864807, -2.190968227098594
LBFGS Accumulation History: 3 points
Removed measurement 54a65417 to history. Total: 3
th(0)=-216.0169503055568;dx=-0.003432505379856581
Armijo: th(105000.00000000004)=-167.85609432661485; dx=4.179924741299081E-4 evalInputDelta=-48.16085597894195
New Minimum: -216.0169503055568 > -234.96632725081187
WOLF (strong): th(52500.00000000002)=-234.96632725081187; dx=2.0523609387137635E-4 evalInputDelta=18.949376945255068
New Minimum: -234.96632725081187 > -248.41931857780193
END: th(17500.000000000007)=-248.41931857780193; dx=-0.0010968336268117505 evalInputDelta=32.40236827224513
Fitness changed from -216.0169503055568 to -248.41931857780193
Iteration 9 complete. Error: -248.41931857780193 Total: 0.6628; Orientation: 0.0311; Line Search: 0.5051
Adding measurement 5e7ce221 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.834e+04, gradient 5.077e-02, dot -0.196; [5b66a2e5-0553-4367-adfb-fa22b5b08dda = 1.000/1.000e+00]
Orientation rejected. Popping history element from -248.41931857780193, -42.31079252979662, -17.942207758864807, -2.190968227098594
LBFGS Accumulation History: 3 points
Removed measurement 5e7ce221 to history. Total: 3
th(0)=-248.41931857780193;dx=-0.0023245646422183613
New Minimum: -248.41931857780193 > -261.8814716834846
END: th(37702.607075557986)=-261.8814716834846; dx=-4.961793461140209E-4 evalInputDelta=13.462153105682688
Fitness changed from -248.41931857780193 to -261.8814716834846
Iteration 10 complete. Error: -261.8814716834846 Total: 0.3712; Orientation: 0.0285; Line Search: 0.2328
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 6a44481b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.356e+04, gradient 5.122e-02, dot -0.209; [5b66a2e5-0553-4367-adfb-fa22b5b08dda = 1.000/1.000e+00]
Orientation rejected. Popping history element from -261.8814716834846, -42.31079252979662, -17.942207758864807, -2.190968227098594
LBFGS Accumulation History: 3 points
Removed measurement 6a44481b to history. Total: 3
th(0)=-261.8814716834846;dx=-0.002442116507440843
Armijo: th(81227.80458822368)=-201.8435871749521; dx=3.765096262426487E-4 evalInputDelta=-60.03788450853253
Armijo: th(40613.90229411184)=-256.2497928859765; dx=-1.7190099211791699E-4 evalInputDelta=-5.63167879750813
New Minimum: -261.8814716834846 > -269.9000020859896
END: th(13537.967431370613)=-269.9000020859896; dx=-9.397697999651109E-4 evalInputDelta=8.018530402505007
Fitness changed from -261.8814716834846 to -269.9000020859896
Iteration 11 complete. Error: -269.9000020859896 Total: 0.6611; Orientation: 0.0314; Line Search: 0.4977
Adding measurement 7e1b857f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.917e+04, gradient 4.429e-02, dot -0.265; [5b66a2e5-0553-4367-adfb-fa22b5b08dda = 1.000/1.000e+00]
Orientation rejected. Popping history element from -269.9000020859896, -42.31079252979662, -17.942207758864807, -2.190968227098594
LBFGS Accumulation History: 3 points
Removed measurement 7e1b857f to history. Total: 3
th(0)=-269.9000020859896;dx=-0.0018070436970575937
New Minimum: -269.9000020859896 > -275.1892619888587
END: th(29166.666666666686)=-275.1892619888587; dx=-6.120314193615432E-4 evalInputDelta=5.289259902869048
Fitness changed from -269.9000020859896 to -275.1892619888587
Iteration 12 complete. Error: -275.1892619888587 Total: 0.3773; Orientation: 0.0296; Line Search: 0.2297
Adding measurement 3fe293f9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.541e+04, gradient 4.836e-02, dot -0.220; [5b66a2e5-0553-4367-adfb-fa22b5b08dda = 1.000/1.000e+00]
Orientation rejected. Popping history element from -275.1892619888587, -42.31079252979662, -17.942207758864807, -2.190968227098594
LBFGS Accumulation History: 3 points
Removed measurement 3fe293f9 to history. Total: 3
th(0)=-275.1892619888587;dx=-0.002176005216312246
Armijo: th(62837.67845926332)=-239.09842352918682; dx=8.68344782537473E-5 evalInputDelta=-36.09083845967186
Armijo: th(31418.83922963166)=-269.7822706858209; dx=-2.833848546100298E-4 evalInputDelta=-5.406991303037785
New Minimum: -275.1892619888587 > -278.0295299034458
END: th(10472.94640987722)=-278.0295299034458; dx=-9.136003378936624E-4 evalInputDelta=2.840267914587116
Fitness changed from -275.1892619888587 to -278.0295299034458
Iteration 13 complete. Error: -278.0295299034458 Total: 0.6096; Orientation: 0.0283; Line Search: 0.4692
Adding measurement 5d9d47e4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.275e+04, gradient 4.451e-02, dot -0.264; [5b66a2e5-0553-4367-adfb-fa22b5b08dda = 1.000/1.000e+00]
Orientation rejected. Popping history element from -278.0295299034458, -42.31079252979662, -17.942207758864807, -2.190968227098594
LBFGS Accumulation History: 3 points
Removed measurement 5d9d47e4 to history. Total: 3
th(0)=-278.0295299034458;dx=-0.001807100262289116
Armijo: th(22563.27905228436)=-277.7216314910447; dx=-6.468596123584561E-4 evalInputDelta=-0.3078984124010731
New Minimum: -278.0295299034458 > -279.3590418599508
END: th(11281.63952614218)=-279.3590418599508; dx=-8.395108224239339E-4 evalInputDelta=1.32951195650503
Fitness changed from -278.0295299034458 to -279.3590418599508
Iteration 14 complete. Error: -279.3590418599508 Total: 0.5267; Orientation: 0.0290; Line Search: 0.3878
Adding measurement 3b7ebf7a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.816e+04, gradient 4.525e-02, dot -0.278; [5b66a2e5-0553-4367-adfb-fa22b5b08dda = 1.000/1.000e+00]
Orientation rejected. Popping history element from -279.3590418599508, -42.31079252979662, -17.942207758864807, -2.190968227098594
LBFGS Accumulation History: 3 points
Removed measurement 3b7ebf7a to history. Total: 3
th(0)=-279.3590418599508;dx=-0.0019180518981318103
Armijo: th(24305.555555555577)=-277.7389697254641; dx=-6.212620926022232E-4 evalInputDelta=-1.6200721344867475
New Minimum: -279.3590418599508 > -279.8547252793128
END: th(12152.777777777788)=-279.8547252793128; dx=-8.510159762985276E-4 evalInputDelta=0.495683419361967
Fitness changed from -279.3590418599508 to -279.8547252793128
Iteration 15 complete. Error: -279.8547252793128 Total: 1.2377; Orientation: 0.0311; Line Search: 1.0858
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -279.8547252793128 (> -Infinity) after 9.149s (< 5400.000s)
Returns
-279.8547252793128