BasicOptimizer.scala:89 executed in 200.03 seconds (2.101 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 13918997869519
Reset training subject: 13923355131386
Adding measurement 18fa1b63 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.20644311606884003;dx=-6.578166299026739E-7
New Minimum: -0.20644311606884003 > -0.2064434289932251
END: th(2.154434690031884)=-0.2064434289932251; dx=-5.889168825674454E-7 evalInputDelta=3.129243850708008E-7
Fitness changed from -0.20644311606884003 to -0.2064434289932251
Iteration 1 complete. Error: -0.2064434289932251 Total: 21.1739; Orientation: 0.0468; Line Search: 8.2823
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 48f60402 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-0.2064434289932251;dx=-5.889066463937061E-7
New Minimum: -0.2064434289932251 > -0.20644502341747284
WOLFE (weak): th(4.641588833612779)=-0.20644502341747284; dx=-5.888596312606989E-7 evalInputDelta=1.5944242477416992E-6
New Minimum: -0.20644502341747284 > -0.2064463496208191
WOLFE (weak): th(9.283177667225559)=-0.2064463496208191; dx=-5.888581106706097E-7 evalInputDelta=2.9206275939941406E-6
New Minimum: -0.2064463496208191 > -0.20645186305046082
WOLFE (weak): th(27.849533001676676)=-0.20645186305046082; dx=-5.888547993474142E-7 evalInputDelta=8.434057235717773E-6
New Minimum: -0.20645186305046082 > -0.20647652447223663
WOLFE (weak): th(111.3981320067067)=-0.20647652447223663; dx=-5.888567205118872E-7 evalInputDelta=3.3095479011535645E-5
New Minimum: -0.20647652447223663 > -0.20660772919654846
WOLFE (weak): th(556.9906600335335)=-0.20660772919654846; dx=-5.887634805443318E-7 evalInputDelta=1.6430020332336426E-4
New Minimum: -0.20660772919654846 > -0.20742715895175934
WOLFE (weak): th(3341.9439602012008)=-0.20742715895175934; dx=-5.881913790969212E-7 evalInputDelta=9.837299585342407E-4
New Minimum: -0.20742715895175934 > -0.21330328285694122
WOLFE (weak): th(23393.607721408407)=-0.21330328285694122; dx=-5.841203854888269E-7 evalInputDelta=0.0068598538637161255
New Minimum: -0.21330328285694122 > -0.25981272757053375
WOLFE (weak): th(187148.86177126726)=-0.25981272757053375; dx=-5.512658780765496E-7 evalInputDelta=0.053369298577308655
New Minimum: -0.25981272757053375 > -0.5580119490623474
END: th(1684339.7559414054)=-0.5580119490623474; dx=-2.6668684966857387E-7 evalInputDelta=0.3515685200691223
Fitness changed from -0.2064434289932251 to -0.5580119490623474
Iteration 2 complete. Error: -0.5580119490623474 Total: 46.8164; Orientation: 0.0447; Line Search: 42.5810
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 7fe194d4 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-0.5580119490623474;dx=-2.747545030753744E-7
New Minimum: -0.5580119490623474 > -0.8791229277849197
END: th(3628800.0000000005)=-0.8791229277849197; dx=-1.1133640300539853E-7 evalInputDelta=0.3211109787225723
Fitness changed from -0.5580119490623474 to -0.8791229277849197
Iteration 3 complete. Error: -0.8791229277849197 Total: 12.9185; Orientation: 0.0452; Line Search: 8.5063
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 338c6a84 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.900e+03, gradient 3.972e-04, dot -0.586; [8475e12a-c2c2-47f4-8b49-1f887fc595df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.8791229277849197, -0.5580119490623474, -0.2064434289932251, -0.20644311606884003
LBFGS Accumulation History: 3 points
Removed measurement 338c6a84 to history. Total: 3
th(0)=-0.8791229277849197;dx=-1.57798738026524E-7
New Minimum: -0.8791229277849197 > -1.1740813553333282
END: th(7818012.603187701)=-1.1740813553333282; dx=-3.130377676534671E-8 evalInputDelta=0.2949584275484085
Fitness changed from -0.8791229277849197 to -1.1740813553333282
Iteration 4 complete. Error: -1.1740813553333282 Total: 13.8079; Orientation: 1.0178; Line Search: 8.5265
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 74264a54 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.706e+03, gradient 3.542e-04, dot -0.381; [8475e12a-c2c2-47f4-8b49-1f887fc595df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.1740813553333282, -0.5580119490623474, -0.2064434289932251, -0.20644311606884003
LBFGS Accumulation History: 3 points
Removed measurement 74264a54 to history. Total: 3
th(0)=-1.1740813553333282;dx=-1.2544887653365082E-7
New Minimum: -1.1740813553333282 > -1.3290306478738785
WOLF (strong): th(1.6843397559414055E7)=-1.3290306478738785; dx=1.8809970888116862E-8 evalInputDelta=0.15494929254055023
New Minimum: -1.3290306478738785 > -1.3831531405448914
END: th(8421698.779707028)=-1.3831531405448914; dx=-8.398499753574753E-10 evalInputDelta=0.2090717852115631
Fitness changed from -1.1740813553333282 to -1.3831531405448914
Iteration 5 complete. Error: -1.3831531405448914 Total: 18.8450; Orientation: 1.0054; Line Search: 13.6499
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 6bcadce8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.028e+03, gradient 3.063e-04, dot -0.347; [8475e12a-c2c2-47f4-8b49-1f887fc595df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.3831531405448914, -0.5580119490623474, -0.2064434289932251, -0.20644311606884003
LBFGS Accumulation History: 3 points
Removed measurement 6bcadce8 to history. Total: 3
th(0)=-1.3831531405448914;dx=-9.3811865039119E-8
New Minimum: -1.3831531405448914 > -1.4398282617330551
WOLF (strong): th(1.8144000000000004E7)=-1.4398282617330551; dx=3.1545255168878287E-8 evalInputDelta=0.05667512118816376
New Minimum: -1.4398282617330551 > -1.5494063347578049
WOLF (strong): th(9072000.000000002)=-1.5494063347578049; dx=4.667769426198667E-9 evalInputDelta=0.1662531942129135
END: th(3024000.0000000005)=-1.4889645874500275; dx=-4.7749196744369374E-8 evalInputDelta=0.10581144690513611
Fitness changed from -1.3831531405448914 to -1.5494063347578049
Iteration 6 complete. Error: -1.5494063347578049 Total: 22.0408; Orientation: 1.0138; Line Search: 16.7452
Adding measurement 40a14526 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.196e+03, gradient 2.522e-04, dot -0.370; [8475e12a-c2c2-47f4-8b49-1f887fc595df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.5494063347578049, -0.5580119490623474, -0.2064434289932251, -0.20644311606884003
LBFGS Accumulation History: 3 points
Removed measurement 40a14526 to history. Total: 3
th(0)=-1.5494063347578049;dx=-6.362079864764816E-8
New Minimum: -1.5494063347578049 > -1.6663530468940735
END: th(6515010.502656418)=-1.6663530468940735; dx=-1.5703735408095876E-8 evalInputDelta=0.11694671213626862
Fitness changed from -1.5494063347578049 to -1.6663530468940735
Iteration 7 complete. Error: -1.6663530468940735 Total: 14.4355; Orientation: 1.0862; Line Search: 8.5415
Adding measurement 635d287f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.556e+03, gradient 1.767e-04, dot -0.538; [8475e12a-c2c2-47f4-8b49-1f887fc595df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.6663530468940735, -0.5580119490623474, -0.2064434289932251, -0.20644311606884003
LBFGS Accumulation History: 3 points
Removed measurement 635d287f to history. Total: 3
th(0)=-1.6663530468940735;dx=-3.120982046273186E-8
New Minimum: -1.6663530468940735 > -1.7752268314361572
END: th(1.4036164632845048E7)=-1.7752268314361572; dx=-2.4164267073651445E-9 evalInputDelta=0.10887378454208374
Fitness changed from -1.6663530468940735 to -1.7752268314361572
Iteration 8 complete. Error: -1.7752268314361572 Total: 13.6204; Orientation: 0.9965; Line Search: 8.4438
Adding measurement 69480dd7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.328e+03, gradient 2.199e-04, dot -0.319; [8475e12a-c2c2-47f4-8b49-1f887fc595df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.7752268314361572, -0.5580119490623474, -0.2064434289932251, -0.20644311606884003
LBFGS Accumulation History: 3 points
Removed measurement 69480dd7 to history. Total: 3
th(0)=-1.7752268314361572;dx=-4.8368750317358656E-8
Armijo: th(3.024000000000001E7)=-1.6847729831933975; dx=1.9940091454690275E-8 evalInputDelta=-0.0904538482427597
New Minimum: -1.7752268314361572 > -1.8158985376358032
WOLF (strong): th(1.5120000000000006E7)=-1.8158985376358032; dx=1.3631846589041169E-8 evalInputDelta=0.040671706199645996
New Minimum: -1.8158985376358032 > -1.8437731862068176
END: th(5040000.000000002)=-1.8437731862068176; dx=-9.862807898699083E-9 evalInputDelta=0.0685463547706604
Fitness changed from -1.7752268314361572 to -1.8437731862068176
Iteration 9 complete. Error: -1.8437731862068176 Total: 22.0553; Orientation: 1.0120; Line Search: 16.9469
Adding measurement 7633cd0b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.632e+03, gradient 1.390e-04, dot -0.538; [8475e12a-c2c2-47f4-8b49-1f887fc595df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.8437731862068176, -0.5580119490623474, -0.2064434289932251, -0.20644311606884003
LBFGS Accumulation History: 3 points
Removed measurement 7633cd0b to history. Total: 3
th(0)=-1.8437731862068176;dx=-1.9309962192899604E-8
New Minimum: -1.8437731862068176 > -1.9089889973402023
END: th(1.0858350837760698E7)=-1.9089889973402023; dx=-5.65702533401326E-9 evalInputDelta=0.0652158111333847
Fitness changed from -1.8437731862068176 to -1.9089889973402023
Iteration 10 complete. Error: -1.9089889973402023 Total: 14.2465; Orientation: 1.0358; Line Search: 9.1189
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -1.9089889973402023 (> -Infinity) after 200.031s (< 5400.000s)
Returns
-1.9089889973402023