BasicOptimizer.scala:89 executed in 65.04 seconds (0.963 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 147356378365900
Reset training subject: 147357270130400
Adding measurement aadedbd to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-2.4109379329006844;dx=-5.438016282994419E-6
New Minimum: -2.4109379329006844 > -2.410950932381327
WOLFE (weak): th(2.154434690031884)=-2.410950932381327; dx=-5.411626585960897E-6 evalInputDelta=1.2999480642594818E-5
New Minimum: -2.410950932381327 > -2.410963799145186
WOLFE (weak): th(4.308869380063768)=-2.410963799145186; dx=-5.412635403851278E-6 evalInputDelta=2.5866244501848712E-5
New Minimum: -2.410963799145186 > -2.4110148675823417
WOLFE (weak): th(12.926608140191302)=-2.4110148675823417; dx=-5.412487774028788E-6 evalInputDelta=7.693468165737016E-5
New Minimum: -2.4110148675823417 > -2.4112459221613296
WOLFE (weak): th(51.70643256076521)=-2.4112459221613296; dx=-5.4046347450980365E-6 evalInputDelta=3.0798926064523613E-4
New Minimum: -2.4112459221613296 > -2.4124760278104507
WOLFE (weak): th(258.53216280382605)=-2.4124760278104507; dx=-5.395943072282486E-6 evalInputDelta=0.0015380949097663077
New Minimum: -2.4124760278104507 > -2.4201538218168883
WOLFE (weak): th(1551.1929768229563)=-2.4201538218168883; dx=-5.400563356416354E-6 evalInputDelta=0.009215888916203951
New Minimum: -2.4201538218168883 > -2.4752556093171307
WOLFE (weak): th(10858.350837760694)=-2.4752556093171307; dx=-5.4144211114244374E-6 evalInputDelta=0.06431767641644637
New Minimum: -2.4752556093171307 > -2.9067032671742936
WOLFE (weak): th(86866.80670208555)=-2.9067032671742936; dx=-5.256890507514538E-6 evalInputDelta=0.49576533427360925
New Minimum: -2.9067032671742936 > -4.806514887928833
END: th(781801.26031877)=-4.806514887928833; dx=-2.3158575216945464E-6 evalInputDelta=2.3955769550281483
Fitness changed from -2.4109379329006844 to -4.806514887928833
Iteration 1 complete. Error: -4.806514887928833 Total: 11.7162; Orientation: 0.0150; Line Search: 9.1456
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 534459b8 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-4.806514887928833;dx=-8.336167517098743E-6
New Minimum: -4.806514887928833 > -8.496217038973597
END: th(1684339.7559414052)=-8.496217038973597; dx=-1.3507615632368197E-6 evalInputDelta=3.6897021510447647
Fitness changed from -4.806514887928833 to -8.496217038973597
Iteration 2 complete. Error: -8.496217038973597 Total: 2.6663; Orientation: 0.0147; Line Search: 1.7264
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 328ecb68 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-8.496217038973597;dx=-9.505160362764581E-6
New Minimum: -8.496217038973597 > -12.893175164146246
END: th(3628800.0)=-12.893175164146246; dx=-1.1157029923069215E-7 evalInputDelta=4.396958125172649
Fitness changed from -8.496217038973597 to -12.893175164146246
Iteration 3 complete. Error: -12.893175164146246 Total: 2.8590; Orientation: 0.0145; Line Search: 1.8882
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 44a45e60 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.504e+04, gradient 2.837e-03, dot -0.186; [8ada09af-23a1-466a-a54d-6e45148a22dd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.893175164146246, -8.496217038973597, -4.806514887928833, -2.4109379329006844
LBFGS Accumulation History: 3 points
Removed measurement 44a45e60 to history. Total: 3
th(0)=-12.893175164146246;dx=-8.036642810549541E-6
New Minimum: -12.893175164146246 > -16.070655673492734
WOLF (strong): th(7818012.6031877)=-16.070655673492734; dx=2.77135546045592E-7 evalInputDelta=3.1774805093464877
New Minimum: -16.070655673492734 > -20.731766606339793
END: th(3909006.30159385)=-20.731766606339793; dx=-4.805999952022812E-7 evalInputDelta=7.838591442193547
Fitness changed from -12.893175164146246 to -20.731766606339793
Iteration 4 complete. Error: -20.731766606339793 Total: 3.7732; Orientation: 0.2875; Line Search: 2.5360
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 241a0275 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.813e+04, gradient 2.741e-03, dot -0.189; [8ada09af-23a1-466a-a54d-6e45148a22dd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -20.731766606339793, -8.496217038973597, -4.806514887928833, -2.4109379329006844
LBFGS Accumulation History: 3 points
Removed measurement 241a0275 to history. Total: 3
th(0)=-20.731766606339793;dx=-7.501930200303834E-6
New Minimum: -20.731766606339793 > -20.900842154965407
WOLF (strong): th(8421698.779707026)=-20.900842154965407; dx=4.193595266465808E-7 evalInputDelta=0.1690755486256137
New Minimum: -20.900842154965407 > -27.59843843816178
END: th(4210849.389853513)=-27.59843843816178; dx=-4.4355460556488114E-7 evalInputDelta=6.866671831821986
Fitness changed from -20.731766606339793 to -27.59843843816178
Iteration 5 complete. Error: -27.59843843816178 Total: 4.0720; Orientation: 0.2909; Line Search: 2.8471
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 3c900fc4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.870e+04, gradient 2.886e-03, dot -0.199; [8ada09af-23a1-466a-a54d-6e45148a22dd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -27.59843843816178, -8.496217038973597, -4.806514887928833, -2.4109379329006844
LBFGS Accumulation History: 3 points
Removed measurement 3c900fc4 to history. Total: 3
th(0)=-27.59843843816178;dx=-8.274522682124957E-6
Armijo: th(9072000.0)=-23.293474282580554; dx=6.58499301661485E-7 evalInputDelta=-4.304964155581224
New Minimum: -27.59843843816178 > -32.805145974933914
WOLF (strong): th(4536000.0)=-32.805145974933914; dx=1.8032506102215442E-7 evalInputDelta=5.206707536772136
New Minimum: -32.805145974933914 > -34.265232891791435
END: th(1512000.0)=-34.265232891791435; dx=-3.037063108666855E-6 evalInputDelta=6.666794453629656
Fitness changed from -27.59843843816178 to -34.265232891791435
Iteration 6 complete. Error: -34.265232891791435 Total: 4.8051; Orientation: 0.2883; Line Search: 3.6003
Adding measurement 43f5cc98 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.203e+04, gradient 2.503e-03, dot -0.253; [8ada09af-23a1-466a-a54d-6e45148a22dd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -34.265232891791435, -8.496217038973597, -4.806514887928833, -2.4109379329006844
LBFGS Accumulation History: 3 points
Removed measurement 43f5cc98 to history. Total: 3
th(0)=-34.265232891791435;dx=-6.239214525680042E-6
New Minimum: -34.265232891791435 > -40.308858455800625
END: th(3257505.2513282085)=-40.308858455800625; dx=-1.4806044863908954E-6 evalInputDelta=6.04362556400919
Fitness changed from -34.265232891791435 to -40.308858455800625
Iteration 7 complete. Error: -40.308858455800625 Total: 2.8796; Orientation: 0.3222; Line Search: 1.7538
Adding measurement 34f285bd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.037e+04, gradient 2.765e-03, dot -0.274; [8ada09af-23a1-466a-a54d-6e45148a22dd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -40.308858455800625, -8.496217038973597, -4.806514887928833, -2.4109379329006844
LBFGS Accumulation History: 3 points
Removed measurement 34f285bd to history. Total: 3
th(0)=-40.308858455800625;dx=-7.634445924942588E-6
Armijo: th(7018082.316422523)=-34.93841139286229; dx=4.528470837638362E-7 evalInputDelta=-5.370447062938332
New Minimum: -40.308858455800625 > -42.585245562469055
END: th(3509041.1582112615)=-42.585245562469055; dx=-2.0983229787080518E-7 evalInputDelta=2.2763871066684302
Fitness changed from -40.308858455800625 to -42.585245562469055
Iteration 8 complete. Error: -42.585245562469055 Total: 3.7877; Orientation: 0.2887; Line Search: 2.6793
Adding measurement 4666b893 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.667e+04, gradient 3.057e-03, dot -0.235; [8ada09af-23a1-466a-a54d-6e45148a22dd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -42.585245562469055, -8.496217038973597, -4.806514887928833, -2.4109379329006844
LBFGS Accumulation History: 3 points
Removed measurement 4666b893 to history. Total: 3
th(0)=-42.585245562469055;dx=-9.331081427130642E-6
Armijo: th(7560000.000000002)=-31.258166148409444; dx=1.2359880048917345E-6 evalInputDelta=-11.327079414059611
New Minimum: -42.585245562469055 > -43.87341899507323
WOLF (strong): th(3780000.000000001)=-43.87341899507323; dx=1.1171854472272885E-6 evalInputDelta=1.2881734326041752
New Minimum: -43.87341899507323 > -48.78273345717674
END: th(1260000.0000000002)=-48.78273345717674; dx=-2.8990349260899604E-6 evalInputDelta=6.197487894707685
Fitness changed from -42.585245562469055 to -48.78273345717674
Iteration 9 complete. Error: -48.78273345717674 Total: 4.7758; Orientation: 0.2641; Line Search: 3.4804
Adding measurement 44e4f7f9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.886e+05, gradient 2.635e-03, dot -0.303; [8ada09af-23a1-466a-a54d-6e45148a22dd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -48.78273345717674, -8.496217038973597, -4.806514887928833, -2.4109379329006844
LBFGS Accumulation History: 3 points
Removed measurement 44e4f7f9 to history. Total: 3
th(0)=-48.78273345717674;dx=-6.897440137964434E-6
New Minimum: -48.78273345717674 > -52.607821291925106
END: th(2714587.709440174)=-52.607821291925106; dx=-8.758879982368078E-7 evalInputDelta=3.8250878347483663
Fitness changed from -48.78273345717674 to -52.607821291925106
Iteration 10 complete. Error: -52.607821291925106 Total: 2.9500; Orientation: 0.2826; Line Search: 1.7926
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 350d789c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.894e+05, gradient 2.992e-03, dot -0.283; [8ada09af-23a1-466a-a54d-6e45148a22dd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -52.607821291925106, -8.496217038973597, -4.806514887928833, -2.4109379329006844
LBFGS Accumulation History: 3 points
Removed measurement 350d789c to history. Total: 3
th(0)=-52.607821291925106;dx=-8.941244609987149E-6
Armijo: th(5848401.930352103)=-43.40419704162728; dx=8.550278306973847E-7 evalInputDelta=-9.203624250297828
Armijo: th(2924200.9651760515)=-52.313281221259544; dx=7.191423684424773E-7 evalInputDelta=-0.29454007066556187
New Minimum: -52.607821291925106 > -56.502295590851574
END: th(974733.6550586838)=-56.502295590851574; dx=-2.1457957538361045E-6 evalInputDelta=3.8944742989264682
Fitness changed from -52.607821291925106 to -56.502295590851574
Iteration 11 complete. Error: -56.502295590851574 Total: 5.2704; Orientation: 0.2874; Line Search: 3.9202
Adding measurement 1f93173e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.194e+06, gradient 2.614e-03, dot -0.333; [8ada09af-23a1-466a-a54d-6e45148a22dd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -56.502295590851574, -8.496217038973597, -4.806514887928833, -2.4109379329006844
LBFGS Accumulation History: 3 points
Removed measurement 1f93173e to history. Total: 3
th(0)=-56.502295590851574;dx=-6.781828794031897E-6
New Minimum: -56.502295590851574 > -58.661203666158144
END: th(2100000.0000000005)=-58.661203666158144; dx=-7.977479589104722E-7 evalInputDelta=2.15890807530657
Fitness changed from -56.502295590851574 to -58.661203666158144
Iteration 12 complete. Error: -58.661203666158144 Total: 3.0280; Orientation: 0.3126; Line Search: 1.8693
Adding measurement 5c6e482f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.275e+08, gradient 3.067e-03, dot -0.284; [8ada09af-23a1-466a-a54d-6e45148a22dd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -58.661203666158144, -8.496217038973597, -4.806514887928833, -2.4109379329006844
LBFGS Accumulation History: 3 points
Removed measurement 5c6e482f to history. Total: 3
th(0)=-58.661203666158144;dx=-9.365981648727751E-6
Armijo: th(4524312.849066957)=-50.11080175314168; dx=7.738212276658046E-7 evalInputDelta=-8.550401913016465
Armijo: th(2262156.4245334785)=-57.62853987508603; dx=9.24059255855562E-7 evalInputDelta=-1.0326637910721175
New Minimum: -58.661203666158144 > -61.96638122378218
END: th(754052.1415111595)=-61.96638122378218; dx=-2.073358214278448E-6 evalInputDelta=3.3051775576240345
Fitness changed from -58.661203666158144 to -61.96638122378218
Iteration 13 complete. Error: -61.96638122378218 Total: 4.8533; Orientation: 0.2948; Line Search: 3.5591
Adding measurement 79e0efa2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.484e+06, gradient 2.639e-03, dot -0.328; [8ada09af-23a1-466a-a54d-6e45148a22dd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -61.96638122378218, -8.496217038973597, -4.806514887928833, -2.4109379329006844
LBFGS Accumulation History: 3 points
Removed measurement 79e0efa2 to history. Total: 3
th(0)=-61.96638122378218;dx=-6.944923710581553E-6
New Minimum: -61.96638122378218 > -63.30005435964247
END: th(1624556.0917644731)=-63.30005435964247; dx=-7.512284253237238E-7 evalInputDelta=1.3336731358602947
Fitness changed from -61.96638122378218 to -63.30005435964247
Iteration 14 complete. Error: -63.30005435964247 Total: 3.0190; Orientation: 0.4088; Line Search: 1.7926
Adding measurement 5536fbd9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.765e+05, gradient 3.073e-03, dot -0.242; [8ada09af-23a1-466a-a54d-6e45148a22dd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -63.30005435964247, -8.496217038973597, -4.806514887928833, -2.4109379329006844
LBFGS Accumulation History: 3 points
Removed measurement 5536fbd9 to history. Total: 3
th(0)=-63.30005435964247;dx=-9.407732340849374E-6
Armijo: th(3500000.0000000014)=-55.822329575519; dx=7.375129709481636E-7 evalInputDelta=-7.477724784123474
Armijo: th(1750000.0000000007)=-62.3100098718291; dx=1.0503632753876809E-6 evalInputDelta=-0.9900444878133712
New Minimum: -63.30005435964247 > -65.88923268633594
END: th(583333.3333333336)=-65.88923268633594; dx=-1.9932747076926166E-6 evalInputDelta=2.589178326693471
Fitness changed from -63.30005435964247 to -65.88923268633594
Iteration 15 complete. Error: -65.88923268633594 Total: 4.4833; Orientation: 0.2776; Line Search: 3.4171
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -65.88923268633594 (> -Infinity) after 65.040s (< 5400.000s)
Returns
-65.88923268633594