BasicOptimizer.scala:89 executed in 84.53 seconds (1.265 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 307726425935400
Reset training subject: 307728157566300
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -140.23904226437256 > -140.29457290015907
th(0)=-140.29457290015907;dx=-3.257299225920308E-5
New Minimum: -140.29457290015907 > -140.2946315812364
END: th(2.154434690031884)=-140.2946315812364; dx=-2.2940140615878297E-5 evalInputDelta=5.868107731998862E-5
Fitness changed from -140.23904226437256 to -140.2946315812364
Iteration 1 complete. Error: -140.2946315812364 Total: 8.1785; Orientation: 0.0392; Line Search: 3.2306
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 46e939ca to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-140.2946315812364;dx=-2.287886471082408E-5
New Minimum: -140.2946315812364 > -140.29475627844414
WOLFE (weak): th(4.641588833612779)=-140.29475627844414; dx=-2.2841970877140646E-5 evalInputDelta=1.2469720775243331E-4
New Minimum: -140.29475627844414 > -140.29487364042078
WOLFE (weak): th(9.283177667225559)=-140.29487364042078; dx=-2.281915551694212E-5 evalInputDelta=2.4205918438724439E-4
New Minimum: -140.29487364042078 > -140.2953430873456
WOLFE (weak): th(27.849533001676676)=-140.2953430873456; dx=-2.2828399345039214E-5 evalInputDelta=7.115061092122232E-4
New Minimum: -140.2953430873456 > -140.2974629140538
WOLFE (weak): th(111.3981320067067)=-140.2974629140538; dx=-2.2792743161318398E-5 evalInputDelta=0.0028313328174078833
New Minimum: -140.2974629140538 > -140.3087510020103
WOLFE (weak): th(556.9906600335335)=-140.3087510020103; dx=-2.2809402788807348E-5 evalInputDelta=0.014119420773909042
New Minimum: -140.3087510020103 > -140.3793414652232
WOLFE (weak): th(3341.9439602012008)=-140.3793414652232; dx=-2.2816416071360043E-5 evalInputDelta=0.08470988398681811
New Minimum: -140.3793414652232 > -140.88702787422218
WOLFE (weak): th(23393.607721408407)=-140.88702787422218; dx=-2.2757207333979794E-5 evalInputDelta=0.5923962929857964
New Minimum: -140.88702787422218 > -144.9776485578241
WOLFE (weak): th(187148.86177126726)=-144.9776485578241; dx=-2.2227548459105696E-5 evalInputDelta=4.683016976587709
New Minimum: -144.9776485578241 > -175.12218591500329
END: th(1684339.7559414054)=-175.12218591500329; dx=-1.3649845841475018E-5 evalInputDelta=34.8275543337669
Fitness changed from -140.2946315812364 to -175.12218591500329
Iteration 2 complete. Error: -175.12218591500329 Total: 18.2586; Orientation: 0.0606; Line Search: 16.1109
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 4bf6a7ad to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-175.12218591500329;dx=-1.7085623744228864E-5
New Minimum: -175.12218591500329 > -216.2896159038436
END: th(3628800.0000000005)=-216.2896159038436; dx=-5.26016946430194E-6 evalInputDelta=41.167429988840325
Fitness changed from -175.12218591500329 to -216.2896159038436
Iteration 3 complete. Error: -216.2896159038436 Total: 5.4607; Orientation: 0.0624; Line Search: 3.2644
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 71b019d8 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-216.2896159038436;dx=-1.1150870413440203E-5
New Minimum: -216.2896159038436 > -244.94365479320106
WOLF (strong): th(7818012.603187701)=-244.94365479320106; dx=5.610819641254172E-7 evalInputDelta=28.65403888935745
END: th(3909006.3015938504)=-242.35268566636114; dx=-2.597917046265331E-6 evalInputDelta=26.06306976251753
Fitness changed from -216.2896159038436 to -244.94365479320106
Iteration 4 complete. Error: -244.94365479320106 Total: 7.0005; Orientation: 0.0800; Line Search: 4.9805
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 7cfd9cc6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.248e+04, gradient 3.900e-03, dot -0.367; [28e4ab59-6627-462b-ae9e-57f66ffb9adc = 1.000/1.000e+00]
Orientation rejected. Popping history element from -244.94365479320106, -216.2896159038436, -175.12218591500329, -140.2946315812364
LBFGS Accumulation History: 3 points
Removed measurement 7cfd9cc6 to history. Total: 3
th(0)=-244.94365479320106;dx=-1.5211663800997409E-5
New Minimum: -244.94365479320106 > -251.28629421563542
WOLF (strong): th(8421698.779707028)=-251.28629421563542; dx=5.68450069074478E-6 evalInputDelta=6.34263942243436
New Minimum: -251.28629421563542 > -273.53066476028783
WOLF (strong): th(4210849.389853514)=-273.53066476028783; dx=1.5192019778463846E-6 evalInputDelta=28.58700996708677
END: th(1403616.4632845046)=-263.7704033278125; dx=-9.109006358737295E-6 evalInputDelta=18.826748534611454
Fitness changed from -244.94365479320106 to -273.53066476028783
Iteration 5 complete. Error: -273.53066476028783 Total: 9.5862; Orientation: 1.0816; Line Search: 6.6561
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 2b8976d6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.221e+04, gradient 3.048e-03, dot -0.376; [28e4ab59-6627-462b-ae9e-57f66ffb9adc = 1.000/1.000e+00]
Orientation rejected. Popping history element from -273.53066476028783, -216.2896159038436, -175.12218591500329, -140.2946315812364
LBFGS Accumulation History: 3 points
Removed measurement 2b8976d6 to history. Total: 3
th(0)=-273.53066476028783;dx=-9.279804310021578E-6
New Minimum: -273.53066476028783 > -290.63268173823633
END: th(3024000.0000000005)=-290.63268173823633; dx=-1.3442286734319622E-6 evalInputDelta=17.102016977948495
Fitness changed from -273.53066476028783 to -290.63268173823633
Iteration 6 complete. Error: -290.63268173823633 Total: 6.3800; Orientation: 1.1042; Line Search: 3.4071
Adding measurement 4b818e58 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.487e+04, gradient 2.211e-03, dot -0.499; [28e4ab59-6627-462b-ae9e-57f66ffb9adc = 1.000/1.000e+00]
Orientation rejected. Popping history element from -290.63268173823633, -216.2896159038436, -175.12218591500329, -140.2946315812364
LBFGS Accumulation History: 3 points
Removed measurement 4b818e58 to history. Total: 3
th(0)=-290.63268173823633;dx=-4.888078436704451E-6
New Minimum: -290.63268173823633 > -302.8287105442589
WOLF (strong): th(6515010.502656418)=-302.8287105442589; dx=3.6081586854197114E-7 evalInputDelta=12.196028806022582
END: th(3257505.251328209)=-301.5094600994666; dx=-1.6299995633461435E-6 evalInputDelta=10.876778361230265
Fitness changed from -290.63268173823633 to -302.8287105442589
Iteration 7 complete. Error: -302.8287105442589 Total: 7.6948; Orientation: 1.0742; Line Search: 5.0498
Adding measurement 7cf0fe09 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.683e+04, gradient 2.930e-03, dot -0.342; [28e4ab59-6627-462b-ae9e-57f66ffb9adc = 1.000/1.000e+00]
Orientation rejected. Popping history element from -302.8287105442589, -216.2896159038436, -175.12218591500329, -140.2946315812364
LBFGS Accumulation History: 3 points
Removed measurement 7cf0fe09 to history. Total: 3
th(0)=-302.8287105442589;dx=-8.561963981527216E-6
Armijo: th(7018082.316422524)=-301.7303701108466; dx=4.575837333697145E-6 evalInputDelta=-1.098340433412318
New Minimum: -302.8287105442589 > -315.94339707980987
WOLF (strong): th(3509041.158211262)=-315.94339707980987; dx=1.2213325643083332E-6 evalInputDelta=13.114686535550959
END: th(1169680.3860704207)=-311.65562347727666; dx=-5.222530388581167E-6 evalInputDelta=8.826912933017752
Fitness changed from -302.8287105442589 to -315.94339707980987
Iteration 8 complete. Error: -315.94339707980987 Total: 9.5932; Orientation: 1.0784; Line Search: 6.9448
Adding measurement 2a528377 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.313e+04, gradient 2.235e-03, dot -0.411; [28e4ab59-6627-462b-ae9e-57f66ffb9adc = 1.000/1.000e+00]
Orientation rejected. Popping history element from -315.94339707980987, -216.2896159038436, -175.12218591500329, -140.2946315812364
LBFGS Accumulation History: 3 points
Removed measurement 2a528377 to history. Total: 3
th(0)=-315.94339707980987;dx=-4.981551810333367E-6
New Minimum: -315.94339707980987 > -324.1165734799182
END: th(2520000.000000001)=-324.1165734799182; dx=-1.1880836345830338E-6 evalInputDelta=8.173176400108332
Fitness changed from -315.94339707980987 to -324.1165734799182
Iteration 9 complete. Error: -324.1165734799182 Total: 6.1501; Orientation: 1.0168; Line Search: 3.6278
Adding measurement 4dbe7e13 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.370e+04, gradient 1.622e-03, dot -0.518; [28e4ab59-6627-462b-ae9e-57f66ffb9adc = 1.000/1.000e+00]
Orientation rejected. Popping history element from -324.1165734799182, -216.2896159038436, -175.12218591500329, -140.2946315812364
LBFGS Accumulation History: 3 points
Removed measurement 4dbe7e13 to history. Total: 3
th(0)=-324.1165734799182;dx=-2.6280457023775705E-6
New Minimum: -324.1165734799182 > -332.22025621197423
END: th(5429175.418880349)=-332.22025621197423; dx=-5.888574027118776E-7 evalInputDelta=8.10368273205603
Fitness changed from -324.1165734799182 to -332.22025621197423
Iteration 10 complete. Error: -332.22025621197423 Total: 5.8743; Orientation: 1.0188; Line Search: 3.2846
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -332.22025621197423 (> -Infinity) after 84.533s (< 5400.000s)
Returns
-332.22025621197423