BasicOptimizer.scala:89 executed in 153.45 seconds (2.609 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 458439851595900
Reset training subject: 458442722849500
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -103.11362379834338 > -103.13105486753923
th(0)=-103.13105486753923;dx=-1.5240628032653524E-5
New Minimum: -103.13105486753923 > -103.13108180663326
END: th(2.154434690031884)=-103.13108180663326; dx=-1.2245357670872675E-5 evalInputDelta=2.69390940275116E-5
Fitness changed from -103.11362379834338 to -103.13108180663326
Iteration 1 complete. Error: -103.13108180663326 Total: 14.1894; Orientation: 0.0704; Line Search: 5.6693
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 60afd5a5 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-103.13108180663326;dx=-1.2295851792676176E-5
New Minimum: -103.13108180663326 > -103.13114107261534
WOLFE (weak): th(4.641588833612779)=-103.13114107261534; dx=-1.225946682824827E-5 evalInputDelta=5.926598207395273E-5
New Minimum: -103.13114107261534 > -103.13120572637511
WOLFE (weak): th(9.283177667225559)=-103.13120572637511; dx=-1.2292039573404233E-5 evalInputDelta=1.2391974185277377E-4
New Minimum: -103.13120572637511 > -103.13144278981423
WOLFE (weak): th(27.849533001676676)=-103.13144278981423; dx=-1.2260179422162292E-5 evalInputDelta=3.609831809683328E-4
New Minimum: -103.13144278981423 > -103.1325203440333
WOLFE (weak): th(111.3981320067067)=-103.1325203440333; dx=-1.2266426639651575E-5 evalInputDelta=0.001438537400034079
New Minimum: -103.1325203440333 > -103.13827429299117
WOLFE (weak): th(556.9906600335335)=-103.13827429299117; dx=-1.2274376703961283E-5 evalInputDelta=0.007192486357908479
New Minimum: -103.13827429299117 > -103.17422381817212
WOLFE (weak): th(3341.9439602012008)=-103.17422381817212; dx=-1.2269267531703715E-5 evalInputDelta=0.04314201153886188
New Minimum: -103.17422381817212 > -103.43244551152786
WOLFE (weak): th(23393.607721408407)=-103.43244551152786; dx=-1.2210145227555445E-5 evalInputDelta=0.3013637048945981
New Minimum: -103.43244551152786 > -105.49943562235751
WOLFE (weak): th(187148.86177126726)=-105.49943562235751; dx=-1.1736127726905162E-5 evalInputDelta=2.368353815724248
New Minimum: -105.49943562235751 > -120.64864175744266
END: th(1684339.7559414054)=-120.64864175744266; dx=-7.555277376623393E-6 evalInputDelta=17.5175599508094
Fitness changed from -103.13108180663326 to -120.64864175744266
Iteration 2 complete. Error: -120.64864175744266 Total: 33.5284; Orientation: 0.1131; Line Search: 29.7070
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 10b50076 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-120.64864175744266;dx=-7.760378708564471E-6
New Minimum: -120.64864175744266 > -140.05092095044014
END: th(3628800.0000000005)=-140.05092095044014; dx=-3.249228090870871E-6 evalInputDelta=19.402279192997483
Fitness changed from -120.64864175744266 to -140.05092095044014
Iteration 3 complete. Error: -140.05092095044014 Total: 9.4422; Orientation: 0.1090; Line Search: 5.6616
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 7c6fa907 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-140.05092095044014;dx=-5.001814069998643E-6
New Minimum: -140.05092095044014 > -157.75507683571308
END: th(7818012.603187701)=-157.75507683571308; dx=-6.906638056698922E-7 evalInputDelta=17.70415588527294
Fitness changed from -140.05092095044014 to -157.75507683571308
Iteration 4 complete. Error: -157.75507683571308 Total: 9.5683; Orientation: 0.1095; Line Search: 5.9665
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 173aef49 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.699e+04, gradient 2.475e-03, dot -0.484; [84777fe1-c145-46cf-b2ac-980aa0bc2e82 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -157.75507683571308, -140.05092095044014, -120.64864175744266, -103.13108180663326
LBFGS Accumulation History: 3 points
Removed measurement 173aef49 to history. Total: 3
th(0)=-157.75507683571308;dx=-6.125579219724741E-6
Armijo: th(1.6843397559414055E7)=-152.3499091399262; dx=1.7057112590338283E-6 evalInputDelta=-5.405167695786872
New Minimum: -157.75507683571308 > -168.73777531039744
WOLF (strong): th(8421698.779707028)=-168.73777531039744; dx=1.2716674011164493E-6 evalInputDelta=10.982698474684355
New Minimum: -168.73777531039744 > -169.80941447799478
END: th(2807232.926569009)=-169.80941447799478; dx=-2.2098090597108956E-6 evalInputDelta=12.054337642281695
Fitness changed from -157.75507683571308 to -169.80941447799478
Iteration 5 complete. Error: -169.80941447799478 Total: 17.4652; Orientation: 2.0000; Line Search: 12.0364
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 64b023d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.821e+04, gradient 1.744e-03, dot -0.651; [84777fe1-c145-46cf-b2ac-980aa0bc2e82 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -169.80941447799478, -140.05092095044014, -120.64864175744266, -103.13108180663326
LBFGS Accumulation History: 3 points
Removed measurement 64b023d to history. Total: 3
th(0)=-169.80941447799478;dx=-3.02899719634846E-6
New Minimum: -169.80941447799478 > -180.90290887289885
END: th(6048000.000000001)=-180.90290887289885; dx=-9.565056882757757E-7 evalInputDelta=11.09349439490407
Fitness changed from -169.80941447799478 to -180.90290887289885
Iteration 6 complete. Error: -180.90290887289885 Total: 11.2652; Orientation: 2.0530; Line Search: 5.8946
Adding measurement 65441f52 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.687e+04, gradient 1.899e-03, dot -0.567; [84777fe1-c145-46cf-b2ac-980aa0bc2e82 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -180.90290887289885, -140.05092095044014, -120.64864175744266, -103.13108180663326
LBFGS Accumulation History: 3 points
Removed measurement 65441f52 to history. Total: 3
th(0)=-180.90290887289885;dx=-3.5841833577173514E-6
New Minimum: -180.90290887289885 > -184.03293099751045
WOLF (strong): th(1.3030021005312836E7)=-184.03293099751045; dx=8.323994628075157E-7 evalInputDelta=3.1300221246115996
New Minimum: -184.03293099751045 > -189.035616239363
WOLF (strong): th(6515010.502656418)=-189.035616239363; dx=1.522398800935723E-7 evalInputDelta=8.13270736646416
END: th(2171670.167552139)=-186.83795535237496; dx=-1.7414654087462657E-6 evalInputDelta=5.935046479476114
Fitness changed from -180.90290887289885 to -189.035616239363
Iteration 7 complete. Error: -189.035616239363 Total: 16.5753; Orientation: 1.8467; Line Search: 11.6553
Adding measurement 58ad1347 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.038e+04, gradient 2.114e-03, dot -0.474; [84777fe1-c145-46cf-b2ac-980aa0bc2e82 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -189.035616239363, -140.05092095044014, -120.64864175744266, -103.13108180663326
LBFGS Accumulation History: 3 points
Removed measurement 58ad1347 to history. Total: 3
th(0)=-189.035616239363;dx=-4.460730995338614E-6
New Minimum: -189.035616239363 > -199.4414591183501
END: th(4678721.544281682)=-199.4414591183501; dx=-4.860231031214665E-8 evalInputDelta=10.405842878987102
Fitness changed from -189.035616239363 to -199.4414591183501
Iteration 8 complete. Error: -199.4414591183501 Total: 10.6573; Orientation: 2.0077; Line Search: 5.8910
Adding measurement 628ea54c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.900e+04, gradient 1.710e-03, dot -0.555; [84777fe1-c145-46cf-b2ac-980aa0bc2e82 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -199.4414591183501, -140.05092095044014, -120.64864175744266, -103.13108180663326
LBFGS Accumulation History: 3 points
Removed measurement 628ea54c to history. Total: 3
th(0)=-199.4414591183501;dx=-2.915433366544909E-6
New Minimum: -199.4414591183501 > -204.09288129301348
WOLF (strong): th(1.0080000000000002E7)=-204.09288129301348; dx=7.594479215229548E-7 evalInputDelta=4.651422174663367
New Minimum: -204.09288129301348 > -206.6771361778527
END: th(5040000.000000001)=-206.6771361778527; dx=-2.2295327963241211E-7 evalInputDelta=7.235677059502592
Fitness changed from -199.4414591183501 to -206.6771361778527
Iteration 9 complete. Error: -206.6771361778527 Total: 13.6498; Orientation: 1.8163; Line Search: 8.8152
Adding measurement 28c6f4a5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.593e+04, gradient 1.675e-03, dot -0.550; [84777fe1-c145-46cf-b2ac-980aa0bc2e82 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -206.6771361778527, -140.05092095044014, -120.64864175744266, -103.13108180663326
LBFGS Accumulation History: 3 points
Removed measurement 28c6f4a5 to history. Total: 3
th(0)=-206.6771361778527;dx=-2.8036022903097347E-6
New Minimum: -206.6771361778527 > -209.45604280740892
WOLF (strong): th(1.0858350837760696E7)=-209.45604280740892; dx=8.5260236199669E-7 evalInputDelta=2.778906629556218
New Minimum: -209.45604280740892 > -213.23802802596134
WOLF (strong): th(5429175.418880348)=-213.23802802596134; dx=3.0410305936500406E-8 evalInputDelta=6.560891848108639
END: th(1809725.1396267826)=-210.83677908376595; dx=-1.6606577355507007E-6 evalInputDelta=4.159642905913245
Fitness changed from -206.6771361778527 to -213.23802802596134
Iteration 10 complete. Error: -213.23802802596134 Total: 16.4951; Orientation: 1.8542; Line Search: 11.6713
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -213.23802802596134 (> -Infinity) after 153.451s (< 5400.000s)
Returns
-213.23802802596134