BasicOptimizer.scala:89 executed in 210.64 seconds (2.054 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 3256145598485
Reset training subject: 3260449776435
Adding measurement 179c3f56 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.3205611854791641;dx=-8.391110618177232E-7
New Minimum: -0.3205611854791641 > -0.32056187093257904
WOLFE (weak): th(2.154434690031884)=-0.32056187093257904; dx=-7.899387387216913E-7 evalInputDelta=6.854534149169922E-7
New Minimum: -0.32056187093257904 > -0.32056280970573425
WOLFE (weak): th(4.308869380063768)=-0.32056280970573425; dx=-7.899369799225204E-7 evalInputDelta=1.6242265701293945E-6
New Minimum: -0.32056280970573425 > -0.3205663114786148
WOLFE (weak): th(12.926608140191302)=-0.3205663114786148; dx=-7.89926836972033E-7 evalInputDelta=5.125999450683594E-6
New Minimum: -0.3205663114786148 > -0.3205816447734833
WOLFE (weak): th(51.70643256076521)=-0.3205816447734833; dx=-7.899192387435891E-7 evalInputDelta=2.0459294319152832E-5
New Minimum: -0.3205816447734833 > -0.32066333293914795
WOLFE (weak): th(258.53216280382605)=-0.32066333293914795; dx=-7.898771968724909E-7 evalInputDelta=1.0214745998382568E-4
New Minimum: -0.32066333293914795 > -0.3211737424135208
WOLFE (weak): th(1551.1929768229563)=-0.3211737424135208; dx=-7.895204772504565E-7 evalInputDelta=6.125569343566895E-4
New Minimum: -0.3211737424135208 > -0.3248415142297745
WOLFE (weak): th(10858.350837760694)=-0.3248415142297745; dx=-7.869392458001905E-7 evalInputDelta=0.0042803287506103516
New Minimum: -0.3248415142297745 > -0.35433751344680786
WOLFE (weak): th(86866.80670208555)=-0.35433751344680786; dx=-7.652566103177492E-7 evalInputDelta=0.03377632796764374
New Minimum: -0.35433751344680786 > -0.5817026346921921
END: th(781801.26031877)=-0.5817026346921921; dx=-5.434615660599801E-7 evalInputDelta=0.26114144921302795
Fitness changed from -0.3205611854791641 to -0.5817026346921921
Iteration 1 complete. Error: -0.5817026346921921 Total: 55.6536; Orientation: 0.0451; Line Search: 43.3714
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 192aa0ff to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-0.5817026346921921;dx=-4.6912117969155957E-7
New Minimum: -0.5817026346921921 > -0.8881290405988693
END: th(1684339.7559414052)=-0.8881290405988693; dx=-2.736789471106571E-7 evalInputDelta=0.30642640590667725
Fitness changed from -0.5817026346921921 to -0.8881290405988693
Iteration 2 complete. Error: -0.8881290405988693 Total: 13.6403; Orientation: 0.0453; Line Search: 8.8301
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 7ca0581b to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-0.8881290405988693;dx=-2.5544236918450287E-7
New Minimum: -0.8881290405988693 > -1.2235364243388176
END: th(3628800.0)=-1.2235364243388176; dx=-1.299574381693208E-7 evalInputDelta=0.3354073837399483
Fitness changed from -0.8881290405988693 to -1.2235364243388176
Iteration 3 complete. Error: -1.2235364243388176 Total: 13.0123; Orientation: 0.0440; Line Search: 8.5474
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 52a5ef0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.992e+03, gradient 3.914e-04, dot -0.797; [0430a077-b953-400c-ba60-67dc733db258 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.2235364243388176, -0.8881290405988693, -0.5817026346921921, -0.3205611854791641
LBFGS Accumulation History: 3 points
Removed measurement 52a5ef0 to history. Total: 3
th(0)=-1.2235364243388176;dx=-1.5316594234564E-7
New Minimum: -1.2235364243388176 > -1.5504577979445457
END: th(7818012.6031877)=-1.5504577979445457; dx=-4.1763607688828084E-8 evalInputDelta=0.32692137360572815
Fitness changed from -1.2235364243388176 to -1.5504577979445457
Iteration 4 complete. Error: -1.5504577979445457 Total: 13.8389; Orientation: 1.0239; Line Search: 8.3682
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 4c8986e7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.523e+03, gradient 3.488e-04, dot -0.571; [0430a077-b953-400c-ba60-67dc733db258 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.5504577979445457, -0.8881290405988693, -0.5817026346921921, -0.3205611854791641
LBFGS Accumulation History: 3 points
Removed measurement 4c8986e7 to history. Total: 3
th(0)=-1.5504577979445457;dx=-1.2169162045906232E-7
New Minimum: -1.5504577979445457 > -1.696624219417572
WOLF (strong): th(1.684339755941405E7)=-1.696624219417572; dx=1.5809045414775415E-8 evalInputDelta=0.14616642147302628
New Minimum: -1.696624219417572 > -1.7375136762857437
END: th(8421698.779707026)=-1.7375136762857437; dx=-1.0865780103894133E-9 evalInputDelta=0.18705587834119797
Fitness changed from -1.5504577979445457 to -1.7375136762857437
Iteration 5 complete. Error: -1.7375136762857437 Total: 18.2709; Orientation: 1.0013; Line Search: 13.1900
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 6e668561 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.481e+03, gradient 3.151e-04, dot -0.485; [0430a077-b953-400c-ba60-67dc733db258 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.7375136762857437, -0.8881290405988693, -0.5817026346921921, -0.3205611854791641
LBFGS Accumulation History: 3 points
Removed measurement 6e668561 to history. Total: 3
th(0)=-1.7375136762857437;dx=-9.930039852383876E-8
New Minimum: -1.7375136762857437 > -1.7576688528060913
WOLF (strong): th(1.8144E7)=-1.7576688528060913; dx=3.607421910324044E-8 evalInputDelta=0.020155176520347595
New Minimum: -1.7576688528060913 > -1.8899402618408203
WOLF (strong): th(9072000.0)=-1.8899402618408203; dx=1.1297907347877331E-8 evalInputDelta=0.1524265855550766
END: th(3024000.0)=-1.8432993292808533; dx=-4.4318331062237026E-8 evalInputDelta=0.10578565299510956
Fitness changed from -1.7375136762857437 to -1.8899402618408203
Iteration 6 complete. Error: -1.8899402618408203 Total: 22.7239; Orientation: 1.0117; Line Search: 17.5392
Adding measurement 773ee131 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.290e+03, gradient 2.752e-04, dot -0.442; [0430a077-b953-400c-ba60-67dc733db258 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.8899402618408203, -0.8881290405988693, -0.5817026346921921, -0.3205611854791641
LBFGS Accumulation History: 3 points
Removed measurement 773ee131 to history. Total: 3
th(0)=-1.8899402618408203;dx=-7.574890854514283E-8
New Minimum: -1.8899402618408203 > -2.0035399943590164
END: th(6515010.502656417)=-2.0035399943590164; dx=-1.000471873955839E-8 evalInputDelta=0.1135997325181961
Fitness changed from -1.8899402618408203 to -2.0035399943590164
Iteration 7 complete. Error: -2.0035399943590164 Total: 14.0128; Orientation: 0.9952; Line Search: 8.9535
Adding measurement 2f333a6f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.569e+03, gradient 1.896e-04, dot -0.548; [0430a077-b953-400c-ba60-67dc733db258 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.0035399943590164, -0.8881290405988693, -0.5817026346921921, -0.3205611854791641
LBFGS Accumulation History: 3 points
Removed measurement 2f333a6f to history. Total: 3
th(0)=-2.0035399943590164;dx=-3.594332673979397E-8
New Minimum: -2.0035399943590164 > -2.07182639837265
WOLF (strong): th(1.4036164632845046E7)=-2.07182639837265; dx=1.0429157288100301E-8 evalInputDelta=0.06828640401363373
New Minimum: -2.07182639837265 > -2.0788363441824913
END: th(7018082.316422523)=-2.0788363441824913; dx=-7.941638713733918E-9 evalInputDelta=0.07529634982347488
Fitness changed from -2.0035399943590164 to -2.0788363441824913
Iteration 8 complete. Error: -2.0788363441824913 Total: 17.9846; Orientation: 0.9999; Line Search: 12.6691
Adding measurement 7250dca1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.518e+03, gradient 1.757e-04, dot -0.524; [0430a077-b953-400c-ba60-67dc733db258 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.0788363441824913, -0.8881290405988693, -0.5817026346921921, -0.3205611854791641
LBFGS Accumulation History: 3 points
Removed measurement 7250dca1 to history. Total: 3
th(0)=-2.0788363441824913;dx=-3.087343441867116E-8
New Minimum: -2.0788363441824913 > -2.1345404535531998
WOLF (strong): th(1.5120000000000004E7)=-2.1345404535531998; dx=2.5919559498692395E-9 evalInputDelta=0.055704109370708466
END: th(7560000.000000002)=-2.133933626115322; dx=-3.946279735659358E-9 evalInputDelta=0.05509728193283081
Fitness changed from -2.0788363441824913 to -2.1345404535531998
Iteration 9 complete. Error: -2.1345404535531998 Total: 18.6160; Orientation: 0.9859; Line Search: 13.2277
Adding measurement 54936d63 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.153e+03, gradient 2.218e-04, dot -0.373; [0430a077-b953-400c-ba60-67dc733db258 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.1345404535531998, -0.8881290405988693, -0.5817026346921921, -0.3205611854791641
LBFGS Accumulation History: 3 points
Removed measurement 54936d63 to history. Total: 3
th(0)=-2.1345404535531998;dx=-4.919646590143315E-8
Armijo: th(1.6287526256641045E7)=-2.1122176572680473; dx=3.5458285171702175E-8 evalInputDelta=-0.022322796285152435
New Minimum: -2.1345404535531998 > -2.2098726704716682
WOLF (strong): th(8143763.128320523)=-2.2098726704716682; dx=8.322438423063123E-9 evalInputDelta=0.07533221691846848
END: th(2714587.709440174)=-2.1869966834783554; dx=-2.7731161394681176E-8 evalInputDelta=0.05245622992515564
Fitness changed from -2.1345404535531998 to -2.2098726704716682
Iteration 10 complete. Error: -2.2098726704716682 Total: 22.8180; Orientation: 1.0127; Line Search: 17.6756
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -2.2098726704716682 (> -Infinity) after 210.643s (< 5400.000s)
Returns
-2.2098726704716682