BasicOptimizer.scala:89 executed in 62.04 seconds (1.114 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 152549804830200
Reset training subject: 152550743923200
Adding measurement 14e6c7ea to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-211.83659312663616;dx=-0.01655268753431014
New Minimum: -211.83659312663616 > -211.87296274831766
WOLFE (weak): th(2.154434690031884)=-211.87296274831766; dx=-0.015403461513798105 evalInputDelta=0.036369621681501485
New Minimum: -211.87296274831766 > -211.90926161036472
WOLFE (weak): th(4.308869380063768)=-211.90926161036472; dx=-0.015432357836748286 evalInputDelta=0.07266848372856316
New Minimum: -211.90926161036472 > -212.0546620272707
WOLFE (weak): th(12.926608140191302)=-212.0546620272707; dx=-0.015391999290260567 evalInputDelta=0.21806890063453466
New Minimum: -212.0546620272707 > -212.70928113566177
WOLFE (weak): th(51.70643256076521)=-212.70928113566177; dx=-0.015461980764305168 evalInputDelta=0.8726880090256088
New Minimum: -212.70928113566177 > -216.21512543934108
WOLFE (weak): th(258.53216280382605)=-216.21512543934108; dx=-0.015545984470912145 evalInputDelta=4.37853231270492
New Minimum: -216.21512543934108 > -238.39008375192537
WOLFE (weak): th(1551.1929768229563)=-238.39008375192537; dx=-0.015212673514127504 evalInputDelta=26.553490625289214
New Minimum: -238.39008375192537 > -378.3814419669654
END: th(10858.350837760694)=-378.3814419669654; dx=-0.009416533847631295 evalInputDelta=166.54484884032922
Fitness changed from -211.83659312663616 to -378.3814419669654
Iteration 1 complete. Error: -378.3814419669654 Total: 9.8878; Orientation: 0.0145; Line Search: 7.3128
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 1f7b649d to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-378.3814419669654;dx=-0.020972721656622038
New Minimum: -378.3814419669654 > -647.4138456196315
END: th(23393.607721408407)=-647.4138456196315; dx=-0.004386811088816306 evalInputDelta=269.0324036526661
Fitness changed from -378.3814419669654 to -647.4138456196315
Iteration 2 complete. Error: -647.4138456196315 Total: 2.7085; Orientation: 0.0143; Line Search: 1.6958
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 33507a1e to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-647.4138456196315;dx=-0.024651024226795368
New Minimum: -647.4138456196315 > -911.2518175564865
WOLF (strong): th(50400.00000000001)=-911.2518175564865; dx=4.2557836538128103E-4 evalInputDelta=263.83797193685496
END: th(25200.000000000004)=-898.9442418748785; dx=-0.002017485405669059 evalInputDelta=251.53039625524696
Fitness changed from -647.4138456196315 to -911.2518175564865
Iteration 3 complete. Error: -911.2518175564865 Total: 3.6745; Orientation: 0.0143; Line Search: 2.7671
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 44523931 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.318e+04, gradient 1.667e-01, dot -0.240; [2dc7564d-2e17-4b21-9688-a9c4ed3d6e28 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -911.2518175564865, -647.4138456196315, -378.3814419669654, -211.83659312663616
LBFGS Accumulation History: 3 points
Removed measurement 44523931 to history. Total: 3
th(0)=-911.2518175564865;dx=-0.027719177184854768
New Minimum: -911.2518175564865 > -1292.0734305758322
WOLF (strong): th(54291.75418880348)=-1292.0734305758322; dx=0.0015789123758871459 evalInputDelta=380.8216130193457
New Minimum: -1292.0734305758322 > -1301.2279296879544
END: th(27145.87709440174)=-1301.2279296879544; dx=-0.004345812601575969 evalInputDelta=389.97611213146797
Fitness changed from -911.2518175564865 to -1301.2279296879544
Iteration 4 complete. Error: -1301.2279296879544 Total: 3.7565; Orientation: 0.2801; Line Search: 2.5763
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 66462500 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.395e+04, gradient 1.377e-01, dot -0.249; [2dc7564d-2e17-4b21-9688-a9c4ed3d6e28 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1301.2279296879544, -647.4138456196315, -378.3814419669654, -211.83659312663616
LBFGS Accumulation History: 3 points
Removed measurement 66462500 to history. Total: 3
th(0)=-1301.2279296879544;dx=-0.018936290667318882
New Minimum: -1301.2279296879544 > -1566.0963412255326
WOLF (strong): th(58484.01930352103)=-1566.0963412255326; dx=5.997255680729658E-4 evalInputDelta=264.86841153757814
New Minimum: -1566.0963412255326 > -1589.3113288465543
END: th(29242.009651760516)=-1589.3113288465543; dx=-0.004367720926761217 evalInputDelta=288.08339915859983
Fitness changed from -1301.2279296879544 to -1589.3113288465543
Iteration 5 complete. Error: -1589.3113288465543 Total: 3.8790; Orientation: 0.2884; Line Search: 2.6857
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 501d94a0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.968e+04, gradient 1.418e-01, dot -0.291; [2dc7564d-2e17-4b21-9688-a9c4ed3d6e28 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1589.3113288465543, -647.4138456196315, -378.3814419669654, -211.83659312663616
LBFGS Accumulation History: 3 points
Removed measurement 501d94a0 to history. Total: 3
th(0)=-1589.3113288465543;dx=-0.019960666171522312
New Minimum: -1589.3113288465543 > -1772.126194716392
WOLF (strong): th(63000.00000000002)=-1772.126194716392; dx=9.162465019696681E-4 evalInputDelta=182.81486586983783
New Minimum: -1772.126194716392 > -1825.2419702603818
END: th(31500.00000000001)=-1825.2419702603818; dx=-0.0026418428482750593 evalInputDelta=235.93064141382752
Fitness changed from -1589.3113288465543 to -1825.2419702603818
Iteration 6 complete. Error: -1825.2419702603818 Total: 3.8220; Orientation: 0.2840; Line Search: 2.5835
Adding measurement 6a893d4b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.464e+04, gradient 1.544e-01, dot -0.254; [2dc7564d-2e17-4b21-9688-a9c4ed3d6e28 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1825.2419702603818, -647.4138456196315, -378.3814419669654, -211.83659312663616
LBFGS Accumulation History: 3 points
Removed measurement 6a893d4b to history. Total: 3
th(0)=-1825.2419702603818;dx=-0.02366367552298618
New Minimum: -1825.2419702603818 > -1827.8981098518593
WOLF (strong): th(67864.69273600436)=-1827.8981098518593; dx=0.0029135899118249973 evalInputDelta=2.6561395914775403
New Minimum: -1827.8981098518593 > -2014.2105153136304
WOLF (strong): th(33932.34636800218)=-2014.2105153136304; dx=0.0010869541254016942 evalInputDelta=188.9685450532486
END: th(11310.782122667393)=-1999.6190887266505; dx=-0.009109227078029754 evalInputDelta=174.3771184662687
Fitness changed from -1825.2419702603818 to -2014.2105153136304
Iteration 7 complete. Error: -2014.2105153136304 Total: 4.4103; Orientation: 0.2764; Line Search: 3.3421
Adding measurement 2f27ca57 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.255e+04, gradient 1.665e-01, dot -0.253; [2dc7564d-2e17-4b21-9688-a9c4ed3d6e28 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2014.2105153136304, -647.4138456196315, -378.3814419669654, -211.83659312663616
LBFGS Accumulation History: 3 points
Removed measurement 2f27ca57 to history. Total: 3
th(0)=-2014.2105153136304;dx=-0.02753191914059778
New Minimum: -2014.2105153136304 > -2282.403667189483
END: th(24368.3413764671)=-2282.403667189483; dx=-0.0010802393578118463 evalInputDelta=268.19315187585244
Fitness changed from -2014.2105153136304 to -2282.403667189483
Iteration 8 complete. Error: -2282.403667189483 Total: 3.0605; Orientation: 0.3117; Line Search: 1.7976
Adding measurement 39153dfa to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.234e+04, gradient 1.474e-01, dot -0.268; [2dc7564d-2e17-4b21-9688-a9c4ed3d6e28 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2282.403667189483, -647.4138456196315, -378.3814419669654, -211.83659312663616
LBFGS Accumulation History: 3 points
Removed measurement 39153dfa to history. Total: 3
th(0)=-2282.403667189483;dx=-0.021638402712699966
New Minimum: -2282.403667189483 > -2338.744535001632
WOLF (strong): th(52500.00000000002)=-2338.744535001632; dx=0.0032957486212876414 evalInputDelta=56.3408678121491
New Minimum: -2338.744535001632 > -2480.7618789396133
END: th(26250.00000000001)=-2480.7618789396133; dx=-0.0015868721556276666 evalInputDelta=198.35821175013052
Fitness changed from -2282.403667189483 to -2480.7618789396133
Iteration 9 complete. Error: -2480.7618789396133 Total: 3.6294; Orientation: 0.3096; Line Search: 2.5025
Adding measurement 20145611 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.713e+05, gradient 1.489e-01, dot -0.316; [2dc7564d-2e17-4b21-9688-a9c4ed3d6e28 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2480.7618789396133, -647.4138456196315, -378.3814419669654, -211.83659312663616
LBFGS Accumulation History: 3 points
Removed measurement 20145611 to history. Total: 3
th(0)=-2480.7618789396133;dx=-0.022130100615327282
New Minimum: -2480.7618789396133 > -2502.8542706278367
WOLF (strong): th(56553.91061333697)=-2502.8542706278367; dx=0.0027756474145069904 evalInputDelta=22.092391688223415
New Minimum: -2502.8542706278367 > -2641.6688853828746
END: th(28276.955306668486)=-2641.6688853828746; dx=-1.5726102067334778E-4 evalInputDelta=160.90700644326125
Fitness changed from -2480.7618789396133 to -2641.6688853828746
Iteration 10 complete. Error: -2641.6688853828746 Total: 3.6852; Orientation: 0.2841; Line Search: 2.5456
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 68094887 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.407e+05, gradient 1.524e-01, dot -0.274; [2dc7564d-2e17-4b21-9688-a9c4ed3d6e28 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2641.6688853828746, -647.4138456196315, -378.3814419669654, -211.83659312663616
LBFGS Accumulation History: 3 points
Removed measurement 68094887 to history. Total: 3
th(0)=-2641.6688853828746;dx=-0.023126552038003238
Armijo: th(60920.853441167754)=-2505.381208518975; dx=0.005273940685063183 evalInputDelta=-136.28767686389938
New Minimum: -2641.6688853828746 > -2793.6617547584387
WOLF (strong): th(30460.426720583877)=-2793.6617547584387; dx=0.0022384653057310414 evalInputDelta=151.99286937556417
END: th(10153.47557352796)=-2786.9060802258837; dx=-0.009372544416974899 evalInputDelta=145.2371948430091
Fitness changed from -2641.6688853828746 to -2793.6617547584387
Iteration 11 complete. Error: -2793.6617547584387 Total: 4.7743; Orientation: 0.2643; Line Search: 3.4368
Adding measurement 7d11ce69 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.225e+05, gradient 1.653e-01, dot -0.287; [2dc7564d-2e17-4b21-9688-a9c4ed3d6e28 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2793.6617547584387, -647.4138456196315, -378.3814419669654, -211.83659312663616
LBFGS Accumulation History: 3 points
Removed measurement 7d11ce69 to history. Total: 3
th(0)=-2793.6617547584387;dx=-0.027222719617373527
New Minimum: -2793.6617547584387 > -2986.545830888922
WOLF (strong): th(21875.000000000015)=-2986.545830888922; dx=5.468577147676605E-5 evalInputDelta=192.8840761304832
END: th(10937.500000000007)=-2969.498779255516; dx=-0.007922524205178975 evalInputDelta=175.83702449707744
Fitness changed from -2793.6617547584387 to -2986.545830888922
Iteration 12 complete. Error: -2986.545830888922 Total: 3.6088; Orientation: 0.2756; Line Search: 2.5401
Adding measurement 70be49ff to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.020e+05, gradient 1.436e-01, dot -0.305; [2dc7564d-2e17-4b21-9688-a9c4ed3d6e28 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2986.545830888922, -647.4138456196315, -378.3814419669654, -211.83659312663616
LBFGS Accumulation History: 3 points
Removed measurement 70be49ff to history. Total: 3
th(0)=-2986.545830888922;dx=-0.02047066282950054
New Minimum: -2986.545830888922 > -3127.907607331137
END: th(23564.129422223745)=-3127.907607331137; dx=-0.0012510066594525024 evalInputDelta=141.36177644221516
Fitness changed from -2986.545830888922 to -3127.907607331137
Iteration 13 complete. Error: -3127.907607331137 Total: 2.9083; Orientation: 0.2753; Line Search: 1.8275
Adding measurement 1669cfb4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.864e+06, gradient 1.438e-01, dot -0.333; [2dc7564d-2e17-4b21-9688-a9c4ed3d6e28 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3127.907607331137, -647.4138456196315, -378.3814419669654, -211.83659312663616
LBFGS Accumulation History: 3 points
Removed measurement 1669cfb4 to history. Total: 3
th(0)=-3127.907607331137;dx=-0.020588425175378816
New Minimum: -3127.907607331137 > -3134.3324648160733
WOLF (strong): th(50767.37786763981)=-3134.3324648160733; dx=0.0024058967756262766 evalInputDelta=6.424857484936183
New Minimum: -3134.3324648160733 > -3250.3656717360277
END: th(25383.688933819903)=-3250.3656717360277; dx=-7.209027511691068E-4 evalInputDelta=122.45806440489059
Fitness changed from -3127.907607331137 to -3250.3656717360277
Iteration 14 complete. Error: -3250.3656717360277 Total: 3.6720; Orientation: 0.2742; Line Search: 2.5395
Adding measurement 682076e0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.293e+06, gradient 1.452e-01, dot -0.307; [2dc7564d-2e17-4b21-9688-a9c4ed3d6e28 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3250.3656717360277, -647.4138456196315, -378.3814419669654, -211.83659312663616
LBFGS Accumulation History: 3 points
Removed measurement 682076e0 to history. Total: 3
th(0)=-3250.3656717360277;dx=-0.020995642462083595
Armijo: th(54687.500000000044)=-3117.966324385175; dx=0.004442501099812188 evalInputDelta=-132.39934735085262
New Minimum: -3250.3656717360277 > -3346.3054253908144
WOLF (strong): th(27343.750000000022)=-3346.3054253908144; dx=9.915739451516132E-4 evalInputDelta=95.93975365478673
New Minimum: -3346.3054253908144 > -3348.71243913239
END: th(9114.583333333341)=-3348.71243913239; dx=-0.007606562755744276 evalInputDelta=98.34676739636234
Fitness changed from -3250.3656717360277 to -3348.71243913239
Iteration 15 complete. Error: -3348.71243913239 Total: 4.4597; Orientation: 0.2724; Line Search: 3.3592
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -3348.71243913239 (> -Infinity) after 62.036s (< 5400.000s)
Returns
-3348.71243913239