BasicOptimizer.scala:89 executed in 1580.85 seconds (17.803 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 566640334786200
Reset training subject: 566662659215200
Adding measurement 6d1103ac to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-60.13947734498419;dx=-8.567997465358294E-6
New Minimum: -60.13947734498419 > -60.13950496523705
WOLFE (weak): th(2.154434690031884)=-60.13950496523705; dx=-8.5779896563024E-6 evalInputDelta=2.7620252858184813E-5
New Minimum: -60.13950496523705 > -60.13953197682777
WOLFE (weak): th(4.308869380063768)=-60.13953197682777; dx=-8.601188935808338E-6 evalInputDelta=5.463184357523687E-5
New Minimum: -60.13953197682777 > -60.13964750649797
WOLFE (weak): th(12.926608140191302)=-60.13964750649797; dx=-8.57894306469108E-6 evalInputDelta=1.7016151377902133E-4
New Minimum: -60.13964750649797 > -60.140158604896385
WOLFE (weak): th(51.70643256076521)=-60.140158604896385; dx=-8.58783875603778E-6 evalInputDelta=6.812599121914786E-4
New Minimum: -60.140158604896385 > -60.14287721275756
WOLFE (weak): th(258.53216280382605)=-60.14287721275756; dx=-8.57095318157449E-6 evalInputDelta=0.0033998677733677596
New Minimum: -60.14287721275756 > -60.15987668574819
WOLFE (weak): th(1551.1929768229563)=-60.15987668574819; dx=-8.58723132123346E-6 evalInputDelta=0.02039934076399419
New Minimum: -60.15987668574819 > -60.28230169939601
WOLFE (weak): th(10858.350837760694)=-60.28230169939601; dx=-8.568702029054536E-6 evalInputDelta=0.14282435441181462
New Minimum: -60.28230169939601 > -61.282704135189604
WOLFE (weak): th(86866.80670208555)=-61.282704135189604; dx=-8.56540357285448E-6 evalInputDelta=1.1432267902054107
New Minimum: -61.282704135189604 > -70.29419313691665
WOLFE (weak): th(781801.26031877)=-70.29419313691665; dx=-8.132256860756302E-6 evalInputDelta=10.154715791932453
New Minimum: -70.29419313691665 > -106.95512746834831
END: th(7818012.6031877)=-106.95512746834831; dx=-2.052465710993155E-7 evalInputDelta=46.815650123364115
Fitness changed from -60.13947734498419 to -106.95512746834831
Iteration 1 complete. Error: -106.95512746834831 Total: 314.0058; Orientation: 0.1464; Line Search: 247.1575
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 4303aa37 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-106.95512746834831;dx=-1.2121408213716263E-5
New Minimum: -106.95512746834831 > -122.7566473472712
WOLF (strong): th(1.684339755941405E7)=-122.7566473472712; dx=1.4874629732213591E-6 evalInputDelta=15.801519878922889
New Minimum: -122.7566473472712 > -149.525249820056
WOLF (strong): th(8421698.779707026)=-149.525249820056; dx=1.9178734856707897E-6 evalInputDelta=42.57012235170768
END: th(2807232.9265690087)=-146.39526602828437; dx=-5.092424561941052E-6 evalInputDelta=39.44013855993606
Fitness changed from -106.95512746834831 to -149.525249820056
Iteration 2 complete. Error: -149.525249820056 Total: 117.7631; Orientation: 0.1671; Line Search: 93.8501
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 42531581 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-149.525249820056;dx=-1.3492276484029365E-5
New Minimum: -149.525249820056 > -208.0614521220869
WOLF (strong): th(6048000.0)=-208.0614521220869; dx=8.89670856877792E-7 evalInputDelta=58.536202302030915
END: th(3024000.0)=-198.19650402355643; dx=-5.982539633539608E-6 evalInputDelta=48.67125420350044
Fitness changed from -149.525249820056 to -208.0614521220869
Iteration 3 complete. Error: -208.0614521220869 Total: 94.1173; Orientation: 0.1527; Line Search: 67.7674
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 4054084e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.339e+04, gradient 2.911e-03, dot -0.512; [57f10751-7513-48e7-bc07-777d1aea1535 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -208.0614521220869, -149.525249820056, -106.95512746834831, -60.13947734498419
LBFGS Accumulation History: 3 points
Removed measurement 4054084e to history. Total: 3
th(0)=-208.0614521220869;dx=-8.458827530899653E-6
New Minimum: -208.0614521220869 > -241.85938367260826
WOLF (strong): th(6515010.502656417)=-241.85938367260826; dx=9.148317757633176E-7 evalInputDelta=33.797931550521355
END: th(3257505.2513282085)=-238.24747876151983; dx=-2.998985251564504E-6 evalInputDelta=30.186026639432924
Fitness changed from -208.0614521220869 to -241.85938367260826
Iteration 4 complete. Error: -241.85938367260826 Total: 93.4922; Orientation: 2.8902; Line Search: 67.4848
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 57eae221 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.713e+04, gradient 2.767e-03, dot -0.505; [57f10751-7513-48e7-bc07-777d1aea1535 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -241.85938367260826, -149.525249820056, -106.95512746834831, -60.13947734498419
LBFGS Accumulation History: 3 points
Removed measurement 57eae221 to history. Total: 3
th(0)=-241.85938367260826;dx=-7.651513505428282E-6
New Minimum: -241.85938367260826 > -269.62495832422326
WOLF (strong): th(7018082.316422523)=-269.62495832422326; dx=1.5718332368828228E-6 evalInputDelta=27.765574651614997
New Minimum: -269.62495832422326 > -269.99673445443386
END: th(3509041.1582112615)=-269.99673445443386; dx=-2.1664525437502245E-6 evalInputDelta=28.137350781825603
Fitness changed from -241.85938367260826 to -269.99673445443386
Iteration 5 complete. Error: -269.99673445443386 Total: 95.7012; Orientation: 2.8899; Line Search: 68.0560
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 4856a7c0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.325e+04, gradient 1.833e-03, dot -0.654; [57f10751-7513-48e7-bc07-777d1aea1535 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -269.99673445443386, -149.525249820056, -106.95512746834831, -60.13947734498419
LBFGS Accumulation History: 3 points
Removed measurement 4856a7c0 to history. Total: 3
th(0)=-269.99673445443386;dx=-3.3533455327929335E-6
New Minimum: -269.99673445443386 > -296.36182240738225
END: th(7560000.000000002)=-296.36182240738225; dx=-1.3238838933422401E-6 evalInputDelta=26.365087952948386
Fitness changed from -269.99673445443386 to -296.36182240738225
Iteration 6 complete. Error: -296.36182240738225 Total: 73.3876; Orientation: 3.0922; Line Search: 45.1056
Adding measurement 10819c96 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.738e+04, gradient 2.030e-03, dot -0.567; [57f10751-7513-48e7-bc07-777d1aea1535 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -296.36182240738225, -149.525249820056, -106.95512746834831, -60.13947734498419
LBFGS Accumulation History: 3 points
Removed measurement 10819c96 to history. Total: 3
th(0)=-296.36182240738225;dx=-4.1010686366490176E-6
New Minimum: -296.36182240738225 > -306.83134047881254
WOLF (strong): th(1.6287526256641045E7)=-306.83134047881254; dx=8.37202614601701E-7 evalInputDelta=10.469518071430286
New Minimum: -306.83134047881254 > -315.4194295145909
WOLF (strong): th(8143763.128320523)=-315.4194295145909; dx=1.726945997316191E-7 evalInputDelta=19.05760710720864
END: th(2714587.709440174)=-309.629798209725; dx=-2.0518217964360215E-6 evalInputDelta=13.267975802342733
Fitness changed from -296.36182240738225 to -315.4194295145909
Iteration 7 complete. Error: -315.4194295145909 Total: 116.1192; Orientation: 2.8009; Line Search: 90.5751
Adding measurement 38d63aea to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.848e+04, gradient 2.319e-03, dot -0.470; [57f10751-7513-48e7-bc07-777d1aea1535 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -315.4194295145909, -149.525249820056, -106.95512746834831, -60.13947734498419
LBFGS Accumulation History: 3 points
Removed measurement 38d63aea to history. Total: 3
th(0)=-315.4194295145909;dx=-5.314287352706214E-6
New Minimum: -315.4194295145909 > -339.76852346979376
WOLF (strong): th(5848401.930352103)=-339.76852346979376; dx=1.2520185471957274E-7 evalInputDelta=24.349093955202875
END: th(2924200.9651760515)=-334.4429898753538; dx=-2.67254723630295E-6 evalInputDelta=19.02356036076293
Fitness changed from -315.4194295145909 to -339.76852346979376
Iteration 8 complete. Error: -339.76852346979376 Total: 92.0097; Orientation: 2.7441; Line Search: 67.4728
Adding measurement 4cce9fd4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.855e+04, gradient 1.852e-03, dot -0.535; [57f10751-7513-48e7-bc07-777d1aea1535 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -339.76852346979376, -149.525249820056, -106.95512746834831, -60.13947734498419
LBFGS Accumulation History: 3 points
Removed measurement 4cce9fd4 to history. Total: 3
th(0)=-339.76852346979376;dx=-3.421920917104094E-6
New Minimum: -339.76852346979376 > -356.5271423751725
END: th(6300000.000000002)=-356.5271423751725; dx=-1.7072151198833447E-7 evalInputDelta=16.75861890537874
Fitness changed from -339.76852346979376 to -356.5271423751725
Iteration 9 complete. Error: -356.5271423751725 Total: 70.8338; Orientation: 2.7468; Line Search: 45.9358
Adding measurement 47b60e55 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.937e+04, gradient 1.797e-03, dot -0.523; [57f10751-7513-48e7-bc07-777d1aea1535 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -356.5271423751725, -149.525249820056, -106.95512746834831, -60.13947734498419
LBFGS Accumulation History: 3 points
Removed measurement 47b60e55 to history. Total: 3
th(0)=-356.5271423751725;dx=-3.2217470611680644E-6
New Minimum: -356.5271423751725 > -362.98500087071625
WOLF (strong): th(1.3572938547200872E7)=-362.98500087071625; dx=1.1083338477020155E-6 evalInputDelta=6.457858495543746
New Minimum: -362.98500087071625 > -372.019483565
WOLF (strong): th(6786469.273600436)=-372.019483565; dx=7.236258081960385E-8 evalInputDelta=15.49234118982747
END: th(2262156.4245334785)=-366.11031400977276; dx=-1.99730050182012E-6 evalInputDelta=9.583171634600262
Fitness changed from -356.5271423751725 to -372.019483565
Iteration 10 complete. Error: -372.019483565 Total: 115.3765; Orientation: 2.7103; Line Search: 89.9957
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 7d76335b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.065e+05, gradient 1.807e-03, dot -0.500; [57f10751-7513-48e7-bc07-777d1aea1535 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -372.019483565, -149.525249820056, -106.95512746834831, -60.13947734498419
LBFGS Accumulation History: 3 points
Removed measurement 7d76335b to history. Total: 3
th(0)=-372.019483565;dx=-3.263625651058332E-6
New Minimum: -372.019483565 > -386.91948431900244
END: th(4873668.275293419)=-386.91948431900244; dx=-5.5632885219632E-7 evalInputDelta=14.900000754002463
Fitness changed from -372.019483565 to -386.91948431900244
Iteration 11 complete. Error: -386.91948431900244 Total: 70.5584; Orientation: 2.7979; Line Search: 44.5517
Adding measurement 72931b48 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.044e+05, gradient 1.398e-03, dot -0.595; [57f10751-7513-48e7-bc07-777d1aea1535 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -386.91948431900244, -149.525249820056, -106.95512746834831, -60.13947734498419
LBFGS Accumulation History: 3 points
Removed measurement 72931b48 to history. Total: 3
th(0)=-386.91948431900244;dx=-1.950175389021597E-6
New Minimum: -386.91948431900244 > -402.2652669128385
END: th(1.0500000000000004E7)=-402.2652669128385; dx=-2.340983631009648E-7 evalInputDelta=15.345782593836077
Fitness changed from -386.91948431900244 to -402.2652669128385
Iteration 12 complete. Error: -402.2652669128385 Total: 71.3530; Orientation: 2.7263; Line Search: 46.3559
Adding measurement 2f373448 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.117e+05, gradient 1.806e-03, dot -0.456; [57f10751-7513-48e7-bc07-777d1aea1535 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -402.2652669128385, -149.525249820056, -106.95512746834831, -60.13947734498419
LBFGS Accumulation History: 3 points
Removed measurement 2f373448 to history. Total: 3
th(0)=-402.2652669128385;dx=-3.2530348554660735E-6
Armijo: th(2.262156424533479E7)=-377.9883964461272; dx=1.6728136383313555E-6 evalInputDelta=-24.27687046671133
New Minimum: -402.2652669128385 > -410.26740899910214
WOLF (strong): th(1.1310782122667395E7)=-410.26740899910214; dx=1.2275053464555193E-6 evalInputDelta=8.002142086263632
New Minimum: -410.26740899910214 > -416.05434819064914
END: th(3770260.7075557983)=-416.05434819064914; dx=-1.1480621541730234E-6 evalInputDelta=13.789081277810624
Fitness changed from -402.2652669128385 to -416.05434819064914
Iteration 13 complete. Error: -416.05434819064914 Total: 115.3238; Orientation: 2.7195; Line Search: 90.9712
Adding measurement 53298d20 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.062e+05, gradient 1.171e-03, dot -0.630; [57f10751-7513-48e7-bc07-777d1aea1535 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -416.05434819064914, -149.525249820056, -106.95512746834831, -60.13947734498419
LBFGS Accumulation History: 3 points
Removed measurement 53298d20 to history. Total: 3
th(0)=-416.05434819064914;dx=-1.3658964363524698E-6
New Minimum: -416.05434819064914 > -429.5339214897984
END: th(8122780.458822368)=-429.5339214897984; dx=-8.128853231587848E-7 evalInputDelta=13.479573299149251
Fitness changed from -416.05434819064914 to -429.5339214897984
Iteration 14 complete. Error: -429.5339214897984 Total: 70.9418; Orientation: 2.7349; Line Search: 45.4708
Adding measurement 6a2554ce to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.091e+05, gradient 1.235e-03, dot -0.579; [57f10751-7513-48e7-bc07-777d1aea1535 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -429.5339214897984, -149.525249820056, -106.95512746834831, -60.13947734498419
LBFGS Accumulation History: 3 points
Removed measurement 6a2554ce to history. Total: 3
th(0)=-429.5339214897984;dx=-1.5282788562242674E-6
New Minimum: -429.5339214897984 > -443.3283014799648
END: th(1.750000000000001E7)=-443.3283014799648; dx=-2.8845765272450805E-8 evalInputDelta=13.79437999016642
Fitness changed from -429.5339214897984 to -443.3283014799648
Iteration 15 complete. Error: -443.3283014799648 Total: 68.7250; Orientation: 2.7378; Line Search: 43.6467
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -443.3283014799648 (> -Infinity) after 1580.855s (< 5400.000s)
Returns
-443.3283014799648