BasicOptimizer.scala:89 executed in 594.17 seconds (4.480 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 3552719995579
Reset training subject: 3564922514707
Adding measurement 6e21f52d to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-33.88799214363098;dx=-1.4498186031123563E-6
New Minimum: -33.88799214363098 > -33.88799449801445
END: th(2.154434690031884)=-33.88799449801445; dx=-9.845033071998522E-7 evalInputDelta=2.3543834686279297E-6
Fitness changed from -33.88799214363098 to -33.88799449801445
Iteration 1 complete. Error: -33.88799449801445 Total: 61.8427; Orientation: 0.0738; Line Search: 24.5921
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 76e3c329 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-33.88799449801445;dx=-9.778813285430574E-7
New Minimum: -33.88799449801445 > -33.88799974322319
WOLFE (weak): th(4.641588833612779)=-33.88799974322319; dx=-9.77207387791085E-7 evalInputDelta=5.245208740234375E-6
New Minimum: -33.88799974322319 > -33.88800495862961
WOLFE (weak): th(9.283177667225559)=-33.88800495862961; dx=-9.767252125884976E-7 evalInputDelta=1.0460615158081055E-5
New Minimum: -33.88800495862961 > -33.88802671432495
WOLFE (weak): th(27.849533001676676)=-33.88802671432495; dx=-9.753141617394863E-7 evalInputDelta=3.221631050109863E-5
New Minimum: -33.88802671432495 > -33.88812103867531
WOLFE (weak): th(111.3981320067067)=-33.88812103867531; dx=-9.742594050315335E-7 evalInputDelta=1.265406608581543E-4
New Minimum: -33.88812103867531 > -33.88862660527229
WOLFE (weak): th(556.9906600335335)=-33.88862660527229; dx=-9.7384059915248E-7 evalInputDelta=6.321072578430176E-4
New Minimum: -33.88862660527229 > -33.89178907871246
WOLFE (weak): th(3341.9439602012008)=-33.89178907871246; dx=-9.732868358846794E-7 evalInputDelta=0.0037945806980133057
New Minimum: -33.89178907871246 > -33.91450551152229
WOLFE (weak): th(23393.607721408407)=-33.91450551152229; dx=-9.70044711863372E-7 evalInputDelta=0.026511013507843018
New Minimum: -33.91450551152229 > -34.09655722975731
WOLFE (weak): th(187148.86177126726)=-34.09655722975731; dx=-9.350595601771286E-7 evalInputDelta=0.2085627317428589
New Minimum: -34.09655722975731 > -35.39447659254074
END: th(1684339.7559414054)=-35.39447659254074; dx=-5.630149233465356E-7 evalInputDelta=1.506482094526291
Fitness changed from -33.88799449801445 to -35.39447659254074
Iteration 2 complete. Error: -35.39447659254074 Total: 138.3393; Orientation: 0.0750; Line Search: 125.0646
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 1ec31c93 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-35.39447659254074;dx=-7.335645309686713E-7
New Minimum: -35.39447659254074 > -37.217989444732666
END: th(3628800.0000000005)=-37.217989444732666; dx=-3.022561202270201E-7 evalInputDelta=1.823512852191925
Fitness changed from -35.39447659254074 to -37.217989444732666
Iteration 3 complete. Error: -37.217989444732666 Total: 38.2162; Orientation: 0.0735; Line Search: 24.7538
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 3f5a924f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.527e+03, gradient 7.122e-04, dot -0.588; [51d2f9ad-017b-4a8e-8e50-e84b8e3b2efd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -37.217989444732666, -35.39447659254074, -33.88799449801445, -33.88799214363098
LBFGS Accumulation History: 3 points
Removed measurement 3f5a924f to history. Total: 3
th(0)=-37.217989444732666;dx=-5.071403228170872E-7
New Minimum: -37.217989444732666 > -39.29153525829315
END: th(7818012.603187701)=-39.29153525829315; dx=-1.206526540761664E-7 evalInputDelta=2.073545813560486
Fitness changed from -37.217989444732666 to -39.29153525829315
Iteration 4 complete. Error: -39.29153525829315 Total: 39.3982; Orientation: 1.5617; Line Search: 24.9295
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 7a7f1473 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.024e+03, gradient 6.827e-04, dot -0.384; [51d2f9ad-017b-4a8e-8e50-e84b8e3b2efd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -39.29153525829315, -35.39447659254074, -33.88799449801445, -33.88799214363098
LBFGS Accumulation History: 3 points
Removed measurement 7a7f1473 to history. Total: 3
th(0)=-39.29153525829315;dx=-4.6593161957979944E-7
New Minimum: -39.29153525829315 > -40.72823941707611
WOLF (strong): th(1.6843397559414055E7)=-40.72823941707611; dx=1.863638324677924E-8 evalInputDelta=1.436704158782959
New Minimum: -40.72823941707611 > -40.962273210287094
END: th(8421698.779707028)=-40.962273210287094; dx=-7.265514136084205E-9 evalInputDelta=1.6707379519939423
Fitness changed from -39.29153525829315 to -40.962273210287094
Iteration 5 complete. Error: -40.962273210287094 Total: 53.4463; Orientation: 1.5775; Line Search: 38.1981
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 3ca784bd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.882e+03, gradient 6.543e-04, dot -0.340; [51d2f9ad-017b-4a8e-8e50-e84b8e3b2efd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -40.962273210287094, -35.39447659254074, -33.88799449801445, -33.88799214363098
LBFGS Accumulation History: 3 points
Removed measurement 3ca784bd to history. Total: 3
th(0)=-40.962273210287094;dx=-4.2804180165631737E-7
New Minimum: -40.962273210287094 > -41.54984533786774
WOLF (strong): th(1.8144000000000004E7)=-41.54984533786774; dx=7.494206929285551E-8 evalInputDelta=0.5875721275806427
New Minimum: -41.54984533786774 > -42.47186002135277
WOLF (strong): th(9072000.000000002)=-42.47186002135277; dx=4.987098918093773E-8 evalInputDelta=1.5095868110656738
END: th(3024000.0000000005)=-42.09778136014938; dx=-2.1276117582914444E-7 evalInputDelta=1.1355081498622894
Fitness changed from -40.962273210287094 to -42.47186002135277
Iteration 6 complete. Error: -42.47186002135277 Total: 66.3036; Orientation: 1.5966; Line Search: 51.2900
Adding measurement 29b38a1c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.537e+03, gradient 6.030e-04, dot -0.329; [51d2f9ad-017b-4a8e-8e50-e84b8e3b2efd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -42.47186002135277, -35.39447659254074, -33.88799449801445, -33.88799214363098
LBFGS Accumulation History: 3 points
Removed measurement 29b38a1c to history. Total: 3
th(0)=-42.47186002135277;dx=-3.636147432510082E-7
New Minimum: -42.47186002135277 > -43.882093489170074
END: th(6515010.502656418)=-43.882093489170074; dx=-3.474068325413342E-8 evalInputDelta=1.4102334678173065
Fitness changed from -42.47186002135277 to -43.882093489170074
Iteration 7 complete. Error: -43.882093489170074 Total: 39.5900; Orientation: 1.5675; Line Search: 26.2982
Adding measurement d75c63f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.815e+03, gradient 4.179e-04, dot -0.521; [51d2f9ad-017b-4a8e-8e50-e84b8e3b2efd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -43.882093489170074, -35.39447659254074, -33.88799449801445, -33.88799214363098
LBFGS Accumulation History: 3 points
Removed measurement d75c63f to history. Total: 3
th(0)=-43.882093489170074;dx=-1.7462911990261382E-7
New Minimum: -43.882093489170074 > -44.864500522613525
WOLF (strong): th(1.4036164632845048E7)=-44.864500522613525; dx=1.208727770197026E-8 evalInputDelta=0.9824070334434509
END: th(7018082.316422524)=-44.771547347307205; dx=-4.995343512480618E-8 evalInputDelta=0.8894538581371307
Fitness changed from -43.882093489170074 to -44.864500522613525
Iteration 8 complete. Error: -44.864500522613525 Total: 52.4146; Orientation: 1.6032; Line Search: 37.9989
Adding measurement 3d7b92f3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.452e+03, gradient 5.160e-04, dot -0.331; [51d2f9ad-017b-4a8e-8e50-e84b8e3b2efd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -44.864500522613525, -35.39447659254074, -33.88799449801445, -33.88799214363098
LBFGS Accumulation History: 3 points
Removed measurement 3d7b92f3 to history. Total: 3
th(0)=-44.864500522613525;dx=-2.6616772079151756E-7
New Minimum: -44.864500522613525 > -45.41113793849945
WOLF (strong): th(1.5120000000000006E7)=-45.41113793849945; dx=9.871102500039281E-8 evalInputDelta=0.5466374158859253
New Minimum: -45.41113793849945 > -45.991339445114136
END: th(7560000.000000003)=-45.991339445114136; dx=-4.267572969954041E-9 evalInputDelta=1.1268389225006104
Fitness changed from -44.864500522613525 to -45.991339445114136
Iteration 9 complete. Error: -45.991339445114136 Total: 52.4765; Orientation: 1.5680; Line Search: 38.3227
Adding measurement 5dd65df2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.001e+04, gradient 3.542e-04, dot -0.506; [51d2f9ad-017b-4a8e-8e50-e84b8e3b2efd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -45.991339445114136, -35.39447659254074, -33.88799449801445, -33.88799214363098
LBFGS Accumulation History: 3 points
Removed measurement 5dd65df2 to history. Total: 3
th(0)=-45.991339445114136;dx=-1.2546239842749033E-7
New Minimum: -45.991339445114136 > -46.621702432632446
WOLF (strong): th(1.6287526256641047E7)=-46.621702432632446; dx=2.6001546421994416E-8 evalInputDelta=0.6303629875183105
New Minimum: -46.621702432632446 > -46.6823633313179
END: th(8143763.128320524)=-46.6823633313179; dx=-2.603177008818076E-8 evalInputDelta=0.6910238862037659
Fitness changed from -45.991339445114136 to -46.6823633313179
Iteration 10 complete. Error: -46.6823633313179 Total: 51.4412; Orientation: 1.5843; Line Search: 37.9585
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -46.6823633313179 (> -Infinity) after 594.171s (< 5400.000s)
Returns
-46.6823633313179