BasicOptimizer.scala:89 executed in 58.50 seconds (1.638 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 212165878468900
Reset training subject: 212166725451900
Adding measurement 3fb12b09 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=99.57543556559934;dx=-0.029722888805966637
New Minimum: 99.57543556559934 > 99.53021329883893
WOLFE (weak): th(2.154434690031884)=99.53021329883893; dx=-0.030245482126016122 evalInputDelta=0.04522226676040475
New Minimum: 99.53021329883893 > 99.4851217086525
WOLFE (weak): th(4.308869380063768)=99.4851217086525; dx=-0.028329244535665057 evalInputDelta=0.09031385694683536
New Minimum: 99.4851217086525 > 99.30529171164372
WOLFE (weak): th(12.926608140191302)=99.30529171164372; dx=-0.029596160572832125 evalInputDelta=0.27014385395561646
New Minimum: 99.30529171164372 > 98.48776736369734
WOLFE (weak): th(51.70643256076521)=98.48776736369734; dx=-0.030323745132327082 evalInputDelta=1.087668201901991
New Minimum: 98.48776736369734 > 94.01367380055294
WOLFE (weak): th(258.53216280382605)=94.01367380055294; dx=-0.028486144609608927 evalInputDelta=5.561761765046398
New Minimum: 94.01367380055294 > 68.10210639976503
END: th(1551.1929768229563)=68.10210639976503; dx=-0.022813804401612715 evalInputDelta=31.4733291658343
Fitness changed from 99.57543556559934 to 68.10210639976503
Iteration 1 complete. Error: 68.10210639976503 Total: 8.0192; Orientation: 0.0162; Line Search: 5.6632
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement cb7817d to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=68.10210639976503;dx=-0.03247043534856247
New Minimum: 68.10210639976503 > 17.235982719271107
END: th(3341.943960201201)=17.235982719271107; dx=-0.014160276088817838 evalInputDelta=50.86612368049393
Fitness changed from 68.10210639976503 to 17.235982719271107
Iteration 2 complete. Error: 17.235982719271107 Total: 2.3117; Orientation: 0.0163; Line Search: 1.5837
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 2eaaf9f4 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=17.235982719271107;dx=-0.02787336880550446
New Minimum: 17.235982719271107 > -84.05362840096475
END: th(7200.000000000001)=-84.05362840096475; dx=-0.007258080337604728 evalInputDelta=101.28961112023586
Fitness changed from 17.235982719271107 to -84.05362840096475
Iteration 3 complete. Error: -84.05362840096475 Total: 2.7541; Orientation: 0.0178; Line Search: 1.9226
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 54d70e0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.783e+03, gradient 2.149e-01, dot -0.216; [7085ff4a-0f61-4866-bca9-38525d88f6bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -84.05362840096475, 17.235982719271107, 68.10210639976503, 99.57543556559934
LBFGS Accumulation History: 3 points
Removed measurement 54d70e0 to history. Total: 3
th(0)=-84.05362840096475;dx=-0.04490015166646194
New Minimum: -84.05362840096475 > -175.17446817175664
END: th(15511.929768229566)=-175.17446817175664; dx=-0.0020873552591134225 evalInputDelta=91.12083977079189
Fitness changed from -84.05362840096475 to -175.17446817175664
Iteration 4 complete. Error: -175.17446817175664 Total: 2.7902; Orientation: 0.3084; Line Search: 1.7625
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 56a11b02 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.246e+03, gradient 2.081e-01, dot -0.127; [7085ff4a-0f61-4866-bca9-38525d88f6bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -175.17446817175664, 17.235982719271107, 68.10210639976503, 99.57543556559934
LBFGS Accumulation History: 3 points
Removed measurement 56a11b02 to history. Total: 3
th(0)=-175.17446817175664;dx=-0.042463915114408335
New Minimum: -175.17446817175664 > -216.58095403057177
WOLF (strong): th(33419.43960201202)=-216.58095403057177; dx=4.2272917656732033E-4 evalInputDelta=41.40648585881513
New Minimum: -216.58095403057177 > -358.58394247940294
END: th(16709.71980100601)=-358.58394247940294; dx=-0.004676719140409391 evalInputDelta=183.4094743076463
Fitness changed from -175.17446817175664 to -358.58394247940294
Iteration 5 complete. Error: -358.58394247940294 Total: 3.4350; Orientation: 0.3041; Line Search: 2.4213
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 6ec101bf to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.732e+03, gradient 2.094e-01, dot -0.116; [7085ff4a-0f61-4866-bca9-38525d88f6bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -358.58394247940294, 17.235982719271107, 68.10210639976503, 99.57543556559934
LBFGS Accumulation History: 3 points
Removed measurement 6ec101bf to history. Total: 3
th(0)=-358.58394247940294;dx=-0.04332616762551578
Armijo: th(36000.000000000015)=-319.8459854743574; dx=6.944106649541741E-4 evalInputDelta=-38.73795700504553
New Minimum: -358.58394247940294 > -542.0659747819593
END: th(18000.000000000007)=-542.0659747819593; dx=-0.005888547204277788 evalInputDelta=183.48203230255638
Fitness changed from -358.58394247940294 to -542.0659747819593
Iteration 6 complete. Error: -542.0659747819593 Total: 3.4050; Orientation: 0.3033; Line Search: 2.2605
Adding measurement 3bf36f10 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.261e+04, gradient 2.131e-01, dot -0.115; [7085ff4a-0f61-4866-bca9-38525d88f6bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -542.0659747819593, 17.235982719271107, 68.10210639976503, 99.57543556559934
LBFGS Accumulation History: 3 points
Removed measurement 3bf36f10 to history. Total: 3
th(0)=-542.0659747819593;dx=-0.04522105295347248
Armijo: th(38779.824420573925)=-384.7802603811648; dx=0.0016882424362174456 evalInputDelta=-157.28571440079452
New Minimum: -542.0659747819593 > -676.6795579180991
END: th(19389.912210286962)=-676.6795579180991; dx=-0.005201475305553907 evalInputDelta=134.61358313613982
Fitness changed from -542.0659747819593 to -676.6795579180991
Iteration 7 complete. Error: -676.6795579180991 Total: 3.2909; Orientation: 0.3032; Line Search: 2.2881
Adding measurement 6ff23d14 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.630e+04, gradient 2.197e-01, dot -0.115; [7085ff4a-0f61-4866-bca9-38525d88f6bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -676.6795579180991, 17.235982719271107, 68.10210639976503, 99.57543556559934
LBFGS Accumulation History: 3 points
Removed measurement 6ff23d14 to history. Total: 3
th(0)=-676.6795579180991;dx=-0.0480894833472385
Armijo: th(41774.29950251503)=-451.59777389249496; dx=0.0019570053426104287 evalInputDelta=-225.08178402560418
New Minimum: -676.6795579180991 > -808.799734263264
END: th(20887.149751257515)=-808.799734263264; dx=-0.004626572367409843 evalInputDelta=132.12017634516485
Fitness changed from -676.6795579180991 to -808.799734263264
Iteration 8 complete. Error: -808.799734263264 Total: 3.7321; Orientation: 0.3062; Line Search: 2.6449
Adding measurement 1d0207fe to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.367e+05, gradient 2.163e-01, dot -0.121; [7085ff4a-0f61-4866-bca9-38525d88f6bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -808.799734263264, 17.235982719271107, 68.10210639976503, 99.57543556559934
LBFGS Accumulation History: 3 points
Removed measurement 1d0207fe to history. Total: 3
th(0)=-808.799734263264;dx=-0.04654924006835186
Armijo: th(45000.00000000002)=-480.3997127839481; dx=0.0027834398125990596 evalInputDelta=-328.4000214793159
New Minimum: -808.799734263264 > -916.2150394327732
END: th(22500.00000000001)=-916.2150394327732; dx=-0.004374671078243093 evalInputDelta=107.41530516950922
Fitness changed from -808.799734263264 to -916.2150394327732
Iteration 9 complete. Error: -916.2150394327732 Total: 3.4171; Orientation: 0.2930; Line Search: 2.4219
Adding measurement 2083512a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.166e+04, gradient 2.224e-01, dot -0.096; [7085ff4a-0f61-4866-bca9-38525d88f6bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -916.2150394327732, 17.235982719271107, 68.10210639976503, 99.57543556559934
LBFGS Accumulation History: 3 points
Removed measurement 2083512a to history. Total: 3
th(0)=-916.2150394327732;dx=-0.04926698997608548
Armijo: th(48474.78052571741)=-503.3036278520202; dx=0.0029586914480715177 evalInputDelta=-412.911411580753
New Minimum: -916.2150394327732 > -1009.3468476430253
END: th(24237.390262858706)=-1009.3468476430253; dx=-0.004699152308478094 evalInputDelta=93.13180821025207
Fitness changed from -916.2150394327732 to -1009.3468476430253
Iteration 10 complete. Error: -1009.3468476430253 Total: 3.9172; Orientation: 0.3165; Line Search: 2.8063
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 174cf211 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.857e+04, gradient 2.219e-01, dot 0.145; [7085ff4a-0f61-4866-bca9-38525d88f6bd = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-1009.3468476430253;dx=-597.5342832975153
Armijo: th(52217.874378143795)=3453.810752114744; dx=0.08701538450300628 evalInputDelta=-4463.157599757769
Armijo: th(26108.937189071898)=3453.429671673781; dx=0.17398664055472168 evalInputDelta=-4462.776519316806
Armijo: th(8702.979063023966)=3451.700747644416; dx=0.5220142657068189 evalInputDelta=-4461.047595287441
Armijo: th(2175.7447657559915)=3446.4945274243128; dx=2.0888378108310013 evalInputDelta=-4455.841375067338
Armijo: th(435.14895315119827)=3409.140675827758; dx=10.379249508694064 evalInputDelta=-4418.487523470783
Armijo: th(72.52482552519972)=3020.159942956595; dx=56.01282437595748 evalInputDelta=-4029.50679059962
Armijo: th(10.360689360742816)=1010.631919407406; dx=127.20263780717598 evalInputDelta=-2019.9787670504313
Armijo: th(1.295086170092852)=-485.53163587270365; dx=191.50403154142643 evalInputDelta=-523.8152117703216
Armijo: th(0.14389846334365022)=-1007.419402532905; dx=-82.73333304630914 evalInputDelta=-1.9274451101202885
New Minimum: -1009.3468476430253 > -1011.8810116448308
END: th(0.014389846334365023)=-1011.8810116448308; dx=-213.66939305738578 evalInputDelta=2.534164001805493
Fitness changed from -1009.3468476430253 to -1011.8810116448308
Iteration 11 complete. Error: -1011.8810116448308 Total: 9.7992; Orientation: 0.2954; Line Search: 8.8049
Adding measurement 23d33dd6 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 3.345e+05, gradient 2.208e-01, dot -0.106; [7085ff4a-0f61-4866-bca9-38525d88f6bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1011.8810116448308, -1009.3468476430253, 17.235982719271107, 68.10210639976503, 99.57543556559934
Rejected: LBFGS Orientation magnitude: 8.188e+05, gradient 2.208e-01, dot -0.112; [7085ff4a-0f61-4866-bca9-38525d88f6bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1011.8810116448308, -1009.3468476430253, 17.235982719271107, 68.10210639976503
LBFGS Accumulation History: 3 points
Removed measurement 23d33dd6 to history. Total: 4
Removed measurement 174cf211 to history. Total: 3
th(0)=-1011.8810116448308;dx=-0.04861624585830383
New Minimum: -1011.8810116448308 > -1011.8819304193835
END: th(0.03100198412698415)=-1011.8819304193835; dx=-0.04009120357865247 evalInputDelta=9.187745527015068E-4
Fitness changed from -1011.8810116448308 to -1011.8819304193835
Iteration 12 complete. Error: -1011.8819304193835 Total: 3.0350; Orientation: 0.6694; Line Search: 1.5699
Adding measurement 5d39e034 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.674e+04, gradient 2.210e-01, dot 0.149; [7085ff4a-0f61-4866-bca9-38525d88f6bd = 1.000/1.000e+00]
th(0)=-1011.8819304193835;dx=-552.2621845836939
New Minimum: -1011.8819304193835 > -1016.0173386703718
END: th(0.06679175006299248)=-1016.0173386703718; dx=-153.07015887824554 evalInputDelta=4.135408250988348
Fitness changed from -1011.8819304193835 to -1016.0173386703718
Iteration 13 complete. Error: -1016.0173386703718 Total: 2.6278; Orientation: 0.2871; Line Search: 1.4287
Adding measurement 2e09ccc6 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.552e+05, gradient 2.173e-01, dot -0.097; [7085ff4a-0f61-4866-bca9-38525d88f6bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1016.0173386703718, -1011.8819304193835, 17.235982719271107, 68.10210639976503, 99.57543556559934
Rejected: LBFGS Orientation magnitude: 3.341e+05, gradient 2.173e-01, dot -0.105; [7085ff4a-0f61-4866-bca9-38525d88f6bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1016.0173386703718, -1011.8819304193835, 17.235982719271107, 68.10210639976503
LBFGS Accumulation History: 3 points
Removed measurement 2e09ccc6 to history. Total: 4
Removed measurement 5d39e034 to history. Total: 3
th(0)=-1016.0173386703718;dx=-0.04706900948338022
New Minimum: -1016.0173386703718 > -1016.0215332475523
END: th(0.14389846334365028)=-1016.0215332475523; dx=-0.037766851760437745 evalInputDelta=0.004194577180442138
Fitness changed from -1016.0173386703718 to -1016.0215332475523
Iteration 14 complete. Error: -1016.0215332475523 Total: 3.0274; Orientation: 0.6696; Line Search: 1.5803
Adding measurement 46a26b70 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.633e+04, gradient 2.175e-01, dot -0.074; [7085ff4a-0f61-4866-bca9-38525d88f6bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1016.0215332475523, 17.235982719271107, 68.10210639976503, 99.57543556559934
LBFGS Accumulation History: 3 points
Removed measurement 46a26b70 to history. Total: 3
th(0)=-1016.0215332475523;dx=-0.04716783464592914
New Minimum: -1016.0215332475523 > -1016.0304754484648
END: th(0.3100198412698416)=-1016.0304754484648; dx=-0.037823609259634114 evalInputDelta=0.00894220091254283
Fitness changed from -1016.0215332475523 to -1016.0304754484648
Iteration 15 complete. Error: -1016.0304754484648 Total: 2.9136; Orientation: 0.3044; Line Search: 1.7992
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -1016.0304754484648 (> -Infinity) after 58.497s (< 5400.000s)
Returns
-1016.0304754484648