BasicOptimizer.scala:89 executed in 129.38 seconds (1.596 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 332224156076900
Reset training subject: 332226927163500
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -101.66978587323236 > -101.87997010164476
th(0)=-101.87997010164476;dx=-2.6595251795076135E-5
New Minimum: -101.87997010164476 > -101.88002216983992
END: th(2.154434690031884)=-101.88002216983992; dx=-2.2594562004418488E-5 evalInputDelta=5.206819515990446E-5
Fitness changed from -101.66978587323236 to -101.88002216983992
Iteration 1 complete. Error: -101.88002216983992 Total: 13.0777; Orientation: 0.0371; Line Search: 5.2889
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 42fd48b1 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-101.88002216983992;dx=-2.2739769584447967E-5
New Minimum: -101.88002216983992 > -101.88014997347884
WOLFE (weak): th(4.641588833612779)=-101.88014997347884; dx=-2.273525910840897E-5 evalInputDelta=1.27803638918067E-4
New Minimum: -101.88014997347884 > -101.88026357657883
WOLFE (weak): th(9.283177667225559)=-101.88026357657883; dx=-2.2727965438029893E-5 evalInputDelta=2.4140673890826747E-4
New Minimum: -101.88026357657883 > -101.88075112177796
WOLFE (weak): th(27.849533001676676)=-101.88075112177796; dx=-2.2717691426200602E-5 evalInputDelta=7.28951938043565E-4
New Minimum: -101.88075112177796 > -101.88293794629907
WOLFE (weak): th(111.3981320067067)=-101.88293794629907; dx=-2.270766839035806E-5 evalInputDelta=0.0029157764591474233
New Minimum: -101.88293794629907 > -101.89460495056211
WOLFE (weak): th(556.9906600335335)=-101.89460495056211; dx=-2.2706262971927223E-5 evalInputDelta=0.014582780722193434
New Minimum: -101.89460495056211 > -101.96746865340059
WOLFE (weak): th(3341.9439602012008)=-101.96746865340059; dx=-2.2689026497810526E-5 evalInputDelta=0.08744648356066875
New Minimum: -101.96746865340059 > -102.49005845676197
WOLFE (weak): th(23393.607721408407)=-102.49005845676197; dx=-2.2525385161209484E-5 evalInputDelta=0.6100362869220533
New Minimum: -102.49005845676197 > -106.60208206192671
WOLFE (weak): th(187148.86177126726)=-106.60208206192671; dx=-2.1033156368150382E-5 evalInputDelta=4.72205989208679
New Minimum: -106.60208206192671 > -130.58046189268248
END: th(1684339.7559414054)=-130.58046189268248; dx=-8.118395317844883E-6 evalInputDelta=28.70043972284256
Fitness changed from -101.88002216983992 to -130.58046189268248
Iteration 2 complete. Error: -130.58046189268248 Total: 29.0045; Orientation: 0.0600; Line Search: 25.8255
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 38c70aed to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-130.58046189268248;dx=-1.18055277184001E-5
New Minimum: -130.58046189268248 > -154.422790811585
END: th(3628800.0000000005)=-154.422790811585; dx=-1.8863233251540584E-6 evalInputDelta=23.842328918902524
Fitness changed from -130.58046189268248 to -154.422790811585
Iteration 3 complete. Error: -154.422790811585 Total: 8.2864; Orientation: 0.0605; Line Search: 5.2779
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 19ce3356 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-154.422790811585;dx=-1.015138150905046E-5
New Minimum: -154.422790811585 > -158.84620602486027
WOLF (strong): th(7818012.603187701)=-158.84620602486027; dx=2.4084087476530706E-6 evalInputDelta=4.423415213275263
New Minimum: -158.84620602486027 > -168.71815193013262
WOLF (strong): th(3909006.3015938504)=-168.71815193013262; dx=7.104044893125614E-7 evalInputDelta=14.295361118547618
END: th(1303002.1005312835)=-165.2038825960126; dx=-4.71158798819699E-6 evalInputDelta=10.781091784427588
Fitness changed from -154.422790811585 to -168.71815193013262
Iteration 4 complete. Error: -168.71815193013262 Total: 13.5387; Orientation: 0.0603; Line Search: 10.5192
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 6403e03 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.246e+04, gradient 3.201e-03, dot -0.544; [8099c9f5-6694-4ea2-a9f4-a2cb7e5739da = 1.000/1.000e+00]
Orientation rejected. Popping history element from -168.71815193013262, -154.422790811585, -130.58046189268248, -101.88002216983992
LBFGS Accumulation History: 3 points
Removed measurement 6403e03 to history. Total: 3
th(0)=-168.71815193013262;dx=-1.0230167954589515E-5
New Minimum: -168.71815193013262 > -184.438911396943
END: th(2807232.926569009)=-184.438911396943; dx=-2.6256274764971874E-7 evalInputDelta=15.720759466810392
Fitness changed from -168.71815193013262 to -184.438911396943
Iteration 5 complete. Error: -184.438911396943 Total: 9.0270; Orientation: 1.0580; Line Search: 4.9876
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 7732204a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.259e+04, gradient 2.495e-03, dot -0.579; [8099c9f5-6694-4ea2-a9f4-a2cb7e5739da = 1.000/1.000e+00]
Orientation rejected. Popping history element from -184.438911396943, -154.422790811585, -130.58046189268248, -101.88002216983992
LBFGS Accumulation History: 3 points
Removed measurement 7732204a to history. Total: 3
th(0)=-184.438911396943;dx=-6.222268345019262E-6
New Minimum: -184.438911396943 > -189.55840098231045
WOLF (strong): th(6048000.000000001)=-189.55840098231045; dx=2.1857538502620605E-6 evalInputDelta=5.1194895853674325
New Minimum: -189.55840098231045 > -194.5287087291977
END: th(3024000.0000000005)=-194.5287087291977; dx=-3.977529949332927E-7 evalInputDelta=10.089797332254676
Fitness changed from -184.438911396943 to -194.5287087291977
Iteration 6 complete. Error: -194.5287087291977 Total: 11.5376; Orientation: 1.0682; Line Search: 7.5509
Adding measurement 59cffc85 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.885e+04, gradient 2.390e-03, dot -0.586; [8099c9f5-6694-4ea2-a9f4-a2cb7e5739da = 1.000/1.000e+00]
Orientation rejected. Popping history element from -194.5287087291977, -154.422790811585, -130.58046189268248, -101.88002216983992
LBFGS Accumulation History: 3 points
Removed measurement 59cffc85 to history. Total: 3
th(0)=-194.5287087291977;dx=-5.700112842202277E-6
New Minimum: -194.5287087291977 > -195.08414487323833
WOLF (strong): th(6515010.502656418)=-195.08414487323833; dx=2.4980082741631464E-6 evalInputDelta=0.555436144040641
New Minimum: -195.08414487323833 > -202.59431120494114
WOLF (strong): th(3257505.251328209)=-202.59431120494114; dx=4.683717264652198E-7 evalInputDelta=8.065602475743447
END: th(1085835.0837760696)=-199.97390510754227; dx=-3.327735582286059E-6 evalInputDelta=5.445196378344576
Fitness changed from -194.5287087291977 to -202.59431120494114
Iteration 7 complete. Error: -202.59431120494114 Total: 13.8416; Orientation: 1.0508; Line Search: 10.4330
Adding measurement 7e013ef8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.330e+04, gradient 2.479e-03, dot -0.517; [8099c9f5-6694-4ea2-a9f4-a2cb7e5739da = 1.000/1.000e+00]
Orientation rejected. Popping history element from -202.59431120494114, -154.422790811585, -130.58046189268248, -101.88002216983992
LBFGS Accumulation History: 3 points
Removed measurement 7e013ef8 to history. Total: 3
th(0)=-202.59431120494114;dx=-6.129788244349629E-6
New Minimum: -202.59431120494114 > -210.58405873987653
END: th(2339360.772140841)=-210.58405873987653; dx=-2.99906543135272E-7 evalInputDelta=7.98974753493539
Fitness changed from -202.59431120494114 to -210.58405873987653
Iteration 8 complete. Error: -210.58405873987653 Total: 8.4859; Orientation: 0.9816; Line Search: 5.0493
Adding measurement 808660b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.796e+04, gradient 1.929e-03, dot -0.633; [8099c9f5-6694-4ea2-a9f4-a2cb7e5739da = 1.000/1.000e+00]
Orientation rejected. Popping history element from -210.58405873987653, -154.422790811585, -130.58046189268248, -101.88002216983992
LBFGS Accumulation History: 3 points
Removed measurement 808660b to history. Total: 3
th(0)=-210.58405873987653;dx=-3.71134196395112E-6
New Minimum: -210.58405873987653 > -215.7202014133726
WOLF (strong): th(5040000.000000001)=-215.7202014133726; dx=8.688368393534996E-7 evalInputDelta=5.136142673496067
New Minimum: -215.7202014133726 > -216.5821489503407
END: th(2520000.0000000005)=-216.5821489503407; dx=-8.883957126396939E-7 evalInputDelta=5.99809021046417
Fitness changed from -210.58405873987653 to -216.5821489503407
Iteration 9 complete. Error: -216.5821489503407 Total: 11.1142; Orientation: 1.0266; Line Search: 7.6280
Adding measurement 58eba89e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.032e+04, gradient 1.800e-03, dot -0.637; [8099c9f5-6694-4ea2-a9f4-a2cb7e5739da = 1.000/1.000e+00]
Orientation rejected. Popping history element from -216.5821489503407, -154.422790811585, -130.58046189268248, -101.88002216983992
LBFGS Accumulation History: 3 points
Removed measurement 58eba89e to history. Total: 3
th(0)=-216.5821489503407;dx=-3.2332223704463934E-6
New Minimum: -216.5821489503407 > -221.07201011115797
WOLF (strong): th(5429175.418880348)=-221.07201011115797; dx=6.015214451275986E-7 evalInputDelta=4.48986116081727
New Minimum: -221.07201011115797 > -221.84191598053582
END: th(2714587.709440174)=-221.84191598053582; dx=-6.582747059769626E-7 evalInputDelta=5.259767030195121
Fitness changed from -216.5821489503407 to -221.84191598053582
Iteration 10 complete. Error: -221.84191598053582 Total: 10.9698; Orientation: 0.9997; Line Search: 7.5492
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -221.84191598053582 (> -Infinity) after 129.384s (< 5400.000s)
Returns
-221.84191598053582