BasicOptimizer.scala:89 executed in 8.86 seconds (0.651 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 151732542143700
Reset training subject: 151732673455400
Adding measurement 1b15b404 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.7442662892325537;dx=-1.1474146757259451E-4
New Minimum: -0.7442662892325537 > -0.7445148487767099
WOLFE (weak): th(2.154434690031884)=-0.7445148487767099; dx=-1.1597022893090336E-4 evalInputDelta=2.4855954415625714E-4
New Minimum: -0.7445148487767099 > -0.7447641887011113
END: th(4.308869380063768)=-0.7447641887011113; dx=-1.0073264285817591E-4 evalInputDelta=4.978994685576055E-4
Fitness changed from -0.7442662892325537 to -0.7447641887011113
Iteration 1 complete. Error: -0.7447641887011113 Total: 0.6902; Orientation: 0.0015; Line Search: 0.3384
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 20b31ff to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-0.7447641887011113;dx=-1.419032023086147E-4
New Minimum: -0.7447641887011113 > -0.7462113351032897
WOLFE (weak): th(9.283177667225559)=-0.7462113351032897; dx=-1.4806252541785177E-4 evalInputDelta=0.0014471464021784675
New Minimum: -0.7462113351032897 > -0.7476494966395274
WOLFE (weak): th(18.566355334451117)=-0.7476494966395274; dx=-1.3502316716211606E-4 evalInputDelta=0.002885307938416126
New Minimum: -0.7476494966395274 > -0.7534265865318376
END: th(55.69906600335335)=-0.7534265865318376; dx=-1.2522400593781172E-4 evalInputDelta=0.00866239783072631
Fitness changed from -0.7447641887011113 to -0.7534265865318376
Iteration 2 complete. Error: -0.7534265865318376 Total: 0.5691; Orientation: 0.0015; Line Search: 0.4459
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 2589ee2a to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-0.7534265865318376;dx=-1.303784853412133E-4
New Minimum: -0.7534265865318376 > -0.7722148052690453
WOLFE (weak): th(120.00000000000001)=-0.7722148052690453; dx=-1.1875369249687222E-4 evalInputDelta=0.018788218737207707
New Minimum: -0.7722148052690453 > -0.7913038817804479
WOLFE (weak): th(240.00000000000003)=-0.7913038817804479; dx=-1.2505967693802604E-4 evalInputDelta=0.037877295248610277
New Minimum: -0.7913038817804479 > -0.8693873381419125
WOLFE (weak): th(720.0000000000001)=-0.8693873381419125; dx=-1.375370402305892E-4 evalInputDelta=0.11596075161007491
New Minimum: -0.8693873381419125 > -1.232589719282373
WOLFE (weak): th(2880.0000000000005)=-1.232589719282373; dx=-1.33453484115442E-4 evalInputDelta=0.47916313275053535
New Minimum: -1.232589719282373 > -2.2903668508337707
END: th(14400.000000000002)=-2.2903668508337707; dx=-2.5519688765228842E-5 evalInputDelta=1.5369402643019332
Fitness changed from -0.7534265865318376 to -2.2903668508337707
Iteration 3 complete. Error: -2.2903668508337707 Total: 0.8086; Orientation: 0.0015; Line Search: 0.6853
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 37416690 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.607e+03, gradient 8.362e-03, dot -0.301; [bae45720-db69-43f3-881a-43993210cc78 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.2903668508337707, -0.7534265865318376, -0.7447641887011113, -0.7442662892325537
LBFGS Accumulation History: 3 points
Removed measurement 37416690 to history. Total: 3
th(0)=-2.2903668508337707;dx=-5.979316106234497E-5
New Minimum: -2.2903668508337707 > -3.4331085672221016
END: th(31023.85953645913)=-3.4331085672221016; dx=-1.3212624477967353E-5 evalInputDelta=1.1427417163883309
Fitness changed from -2.2903668508337707 to -3.4331085672221016
Iteration 4 complete. Error: -3.4331085672221016 Total: 0.3740; Orientation: 0.0277; Line Search: 0.2242
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 608d6b10 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.563e+03, gradient 8.030e-03, dot -0.192; [bae45720-db69-43f3-881a-43993210cc78 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.4331085672221016, -0.7534265865318376, -0.7447641887011113, -0.7442662892325537
LBFGS Accumulation History: 3 points
Removed measurement 608d6b10 to history. Total: 3
th(0)=-3.4331085672221016;dx=-5.915731022467146E-5
New Minimum: -3.4331085672221016 > -4.412464034985884
END: th(66838.87920402404)=-4.412464034985884; dx=-1.3752858986399614E-6 evalInputDelta=0.9793554677637828
Fitness changed from -3.4331085672221016 to -4.412464034985884
Iteration 5 complete. Error: -4.412464034985884 Total: 0.3777; Orientation: 0.0277; Line Search: 0.2256
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 9ff5b73 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.220e+03, gradient 9.494e-03, dot -0.290; [bae45720-db69-43f3-881a-43993210cc78 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.412464034985884, -0.7534265865318376, -0.7447641887011113, -0.7442662892325537
LBFGS Accumulation History: 3 points
Removed measurement 9ff5b73 to history. Total: 3
th(0)=-4.412464034985884;dx=-8.109081186412502E-5
New Minimum: -4.412464034985884 > -5.840193822211505
END: th(144000.00000000006)=-5.840193822211505; dx=-1.3187595108643623E-7 evalInputDelta=1.427729787225621
Fitness changed from -4.412464034985884 to -5.840193822211505
Iteration 6 complete. Error: -5.840193822211505 Total: 0.3825; Orientation: 0.0278; Line Search: 0.2290
Adding measurement 4aaa5281 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.820e+03, gradient 1.302e-02, dot -0.091; [bae45720-db69-43f3-881a-43993210cc78 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.840193822211505, -0.7534265865318376, -0.7447641887011113, -0.7442662892325537
LBFGS Accumulation History: 3 points
Removed measurement 4aaa5281 to history. Total: 3
th(0)=-5.840193822211505;dx=-1.524035839812085E-4
New Minimum: -5.840193822211505 > -10.2409700371178
WOLF (strong): th(310238.5953645914)=-10.2409700371178; dx=1.7518041564323438E-6 evalInputDelta=4.400776214906294
New Minimum: -10.2409700371178 > -10.78578000835379
END: th(155119.2976822957)=-10.78578000835379; dx=-2.9062470875818414E-6 evalInputDelta=4.945586186142284
Fitness changed from -5.840193822211505 to -10.78578000835379
Iteration 7 complete. Error: -10.78578000835379 Total: 0.4827; Orientation: 0.0278; Line Search: 0.3466
Adding measurement 24f348e7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.432e+03, gradient 1.011e-02, dot -0.107; [bae45720-db69-43f3-881a-43993210cc78 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.78578000835379, -0.7534265865318376, -0.7447641887011113, -0.7442662892325537
LBFGS Accumulation History: 3 points
Removed measurement 24f348e7 to history. Total: 3
th(0)=-10.78578000835379;dx=-9.76611539439251E-5
New Minimum: -10.78578000835379 > -17.263740689345877
WOLF (strong): th(334194.39602012024)=-17.263740689345877; dx=5.731765404365244E-6 evalInputDelta=6.477960680992087
New Minimum: -17.263740689345877 > -18.50833233116399
END: th(167097.19801006012)=-18.50833233116399; dx=-7.485509588865532E-6 evalInputDelta=7.722552322810202
Fitness changed from -10.78578000835379 to -18.50833233116399
Iteration 8 complete. Error: -18.50833233116399 Total: 0.4805; Orientation: 0.0285; Line Search: 0.3449
Adding measurement 645adfa3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.100e+04, gradient 1.054e-02, dot -0.262; [bae45720-db69-43f3-881a-43993210cc78 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.50833233116399, -0.7534265865318376, -0.7447641887011113, -0.7442662892325537
LBFGS Accumulation History: 3 points
Removed measurement 645adfa3 to history. Total: 3
th(0)=-18.50833233116399;dx=-1.0161585050670565E-4
New Minimum: -18.50833233116399 > -18.528319116179247
WOLF (strong): th(360000.0000000002)=-18.528319116179247; dx=7.602230248107599E-6 evalInputDelta=0.019986785015255748
New Minimum: -18.528319116179247 > -23.77862377059809
END: th(180000.0000000001)=-23.77862377059809; dx=-9.218663242924934E-7 evalInputDelta=5.2702914394341
Fitness changed from -18.50833233116399 to -23.77862377059809
Iteration 9 complete. Error: -23.77862377059809 Total: 0.4891; Orientation: 0.0290; Line Search: 0.3473
Adding measurement 2cb1ce93 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.787e+04, gradient 1.127e-02, dot -0.223; [bae45720-db69-43f3-881a-43993210cc78 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -23.77862377059809, -0.7534265865318376, -0.7447641887011113, -0.7442662892325537
LBFGS Accumulation History: 3 points
Removed measurement 2cb1ce93 to history. Total: 3
th(0)=-23.77862377059809;dx=-1.1448043851948478E-4
Armijo: th(387798.2442057393)=-20.640368131945454; dx=1.3411365388543452E-5 evalInputDelta=-3.1382556386526375
New Minimum: -23.77862377059809 > -27.329696373944476
WOLF (strong): th(193899.12210286965)=-27.329696373944476; dx=4.132090283589661E-6 evalInputDelta=3.551072603346384
New Minimum: -27.329696373944476 > -29.02182520646039
END: th(64633.040700956546)=-29.02182520646039; dx=-4.163948242184814E-5 evalInputDelta=5.2432014358623
Fitness changed from -23.77862377059809 to -29.02182520646039
Iteration 10 complete. Error: -29.02182520646039 Total: 0.5938; Orientation: 0.0285; Line Search: 0.4561
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement dbb3dd4 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 9.661e+04, gradient 9.488e-03, dot 0.329; [bae45720-db69-43f3-881a-43993210cc78 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-29.02182520646039;dx=-308.11590634167925
Armijo: th(139247.66500838345)=-24.115273182716745; dx=3.7607854393587354E-5 evalInputDelta=-4.906552023743647
Armijo: th(69623.83250419173)=-24.115273182716745; dx=9.334831178992321E-5 evalInputDelta=-4.906552023743647
Armijo: th(23207.94416806391)=-24.115273182716745; dx=2.757891390727658E-4 evalInputDelta=-4.906552023743647
Armijo: th(5801.986042015978)=-24.115297885157187; dx=0.0012968004352265279 evalInputDelta=-4.906527321303205
Armijo: th(1160.3972084031955)=-24.131976657199125; dx=0.006919986619886513 evalInputDelta=-4.889848549261266
Armijo: th(193.39953473386592)=-24.11797209972164; dx=0.03377393104146203 evalInputDelta=-4.903853106738751
Armijo: th(27.628504961980845)=-23.990708532556315; dx=0.2659045405279732 evalInputDelta=-5.0311166739040765
Armijo: th(3.4535631202476056)=-24.26339960942889; dx=1.8931447100481782 evalInputDelta=-4.758425597031501
Armijo: th(0.38372923558306726)=-27.459087037458676; dx=10.95010916232937 evalInputDelta=-1.5627381690017152
New Minimum: -29.02182520646039 > -32.26388173164786
END: th(0.038372923558306724)=-32.26388173164786; dx=-4.286308740381472 evalInputDelta=3.242056525187465
Fitness changed from -29.02182520646039 to -32.26388173164786
Iteration 11 complete. Error: -32.26388173164786 Total: 1.4170; Orientation: 0.0255; Line Search: 1.2681
Adding measurement 7255b64d to history. Total: 4
Rejected: LBFGS Orientation magnitude: 2.689e+06, gradient 9.874e-03, dot -0.198; [bae45720-db69-43f3-881a-43993210cc78 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -32.26388173164786, -29.02182520646039, -0.7534265865318376, -0.7447641887011113, -0.7442662892325537
Rejected: LBFGS Orientation magnitude: 2.349e+06, gradient 9.874e-03, dot -0.309; [bae45720-db69-43f3-881a-43993210cc78 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -32.26388173164786, -29.02182520646039, -0.7534265865318376, -0.7447641887011113
LBFGS Accumulation History: 3 points
Removed measurement 7255b64d to history. Total: 4
Removed measurement dbb3dd4 to history. Total: 3
th(0)=-32.26388173164786;dx=-8.285922637862964E-5
New Minimum: -32.26388173164786 > -32.26388561871477
END: th(0.08267195767195772)=-32.26388561871477; dx=-6.176610962729822E-5 evalInputDelta=3.887066910124304E-6
Fitness changed from -32.26388173164786 to -32.26388561871477
Iteration 12 complete. Error: -32.26388561871477 Total: 0.3917; Orientation: 0.0583; Line Search: 0.2240
Adding measurement 3c1e72cd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.093e+03, gradient 1.016e-02, dot -0.246; [bae45720-db69-43f3-881a-43993210cc78 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -32.26388561871477, -0.7534265865318376, -0.7447641887011113, -0.7442662892325537
LBFGS Accumulation History: 3 points
Removed measurement 3c1e72cd to history. Total: 3
th(0)=-32.26388561871477;dx=-9.733745719979878E-5
New Minimum: -32.26388561871477 > -32.263899223445264
END: th(0.17811133350131325)=-32.263899223445264; dx=-5.9767337877432696E-5 evalInputDelta=1.3604730497718265E-5
Fitness changed from -32.26388561871477 to -32.263899223445264
Iteration 13 complete. Error: -32.263899223445264 Total: 1.0614; Orientation: 0.0297; Line Search: 0.2403
Adding measurement 138bb57a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.065e+03, gradient 1.010e-02, dot -0.248; [bae45720-db69-43f3-881a-43993210cc78 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -32.263899223445264, -0.7534265865318376, -0.7447641887011113, -0.7442662892325537
LBFGS Accumulation History: 3 points
Removed measurement 138bb57a to history. Total: 3
th(0)=-32.263899223445264;dx=-9.067460637188251E-5
New Minimum: -32.263899223445264 > -32.26393226348117
END: th(0.3837292355830673)=-32.26393226348117; dx=-5.88214126690291E-5 evalInputDelta=3.304003590898219E-5
Fitness changed from -32.263899223445264 to -32.26393226348117
Iteration 14 complete. Error: -32.26393226348117 Total: 0.3641; Orientation: 0.0263; Line Search: 0.2280
Adding measurement 4c4176d5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.200e+03, gradient 9.805e-03, dot -0.228; [bae45720-db69-43f3-881a-43993210cc78 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -32.26393226348117, -0.7534265865318376, -0.7447641887011113, -0.7442662892325537
LBFGS Accumulation History: 3 points
Removed measurement 4c4176d5 to history. Total: 3
th(0)=-32.26393226348117;dx=-8.185858874031484E-5
New Minimum: -32.26393226348117 > -32.26399542816152
END: th(0.8267195767195774)=-32.26399542816152; dx=-5.858729837192482E-5 evalInputDelta=6.316468034839318E-5
Fitness changed from -32.26393226348117 to -32.26399542816152
Iteration 15 complete. Error: -32.26399542816152 Total: 0.3630; Orientation: 0.0276; Line Search: 0.2250
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -32.26399542816152 (> -Infinity) after 8.859s (< 5400.000s)
Returns
-32.26399542816152