BasicOptimizer.scala:89 executed in 9.43 seconds (0.776 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 138687476923799
Reset training subject: 138687775508800
Adding measurement 24d8f87a to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.8013160573321375;dx=-7.066181141712431E-5
New Minimum: -0.8013160573321375 > -0.8015490550403663
WOLFE (weak): th(2.154434690031884)=-0.8015490550403663; dx=-7.10550101524139E-5 evalInputDelta=2.329977082288215E-4
New Minimum: -0.8015490550403663 > -0.8017819314262045
WOLFE (weak): th(4.308869380063768)=-0.8017819314262045; dx=-7.123377127030606E-5 evalInputDelta=4.658740940669537E-4
New Minimum: -0.8017819314262045 > -0.8027145553290127
WOLFE (weak): th(12.926608140191302)=-0.8027145553290127; dx=-7.122539539833394E-5 evalInputDelta=0.001398497996875192
New Minimum: -0.8027145553290127 > -0.8069056548731954
WOLFE (weak): th(51.70643256076521)=-0.8069056548731954; dx=-6.940913519809493E-5 evalInputDelta=0.005589597541057922
New Minimum: -0.8069056548731954 > -0.829325246680586
WOLFE (weak): th(258.53216280382605)=-0.829325246680586; dx=-7.06583996091427E-5 evalInputDelta=0.028009189348448538
New Minimum: -0.829325246680586 > -0.9699167232379702
WOLFE (weak): th(1551.1929768229563)=-0.9699167232379702; dx=-6.765468745832776E-5 evalInputDelta=0.16860066590583267
New Minimum: -0.9699167232379702 > -1.7980953180116086
END: th(10858.350837760694)=-1.7980953180116086; dx=-4.456123672789863E-5 evalInputDelta=0.9967792606794711
Fitness changed from -0.8013160573321375 to -1.7980953180116086
Iteration 1 complete. Error: -1.7980953180116086 Total: 1.7921; Orientation: 0.0226; Line Search: 1.1451
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement f01fc6d to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-1.7980953180116086;dx=-5.9647626042580515E-5
New Minimum: -1.7980953180116086 > -3.1170138608555464
END: th(23393.607721408407)=-3.1170138608555464; dx=-2.7665857007818403E-5 evalInputDelta=1.3189185428439378
Fitness changed from -1.7980953180116086 to -3.1170138608555464
Iteration 2 complete. Error: -3.1170138608555464 Total: 0.4245; Orientation: 0.0023; Line Search: 0.2642
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 78508e6b to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-3.1170138608555464;dx=-5.6743182199351826E-5
New Minimum: -3.1170138608555464 > -4.527221648028044
END: th(50400.00000000001)=-4.527221648028044; dx=-1.509518834544189E-5 evalInputDelta=1.4102077871724972
Fitness changed from -3.1170138608555464 to -4.527221648028044
Iteration 3 complete. Error: -4.527221648028044 Total: 0.4177; Orientation: 0.0016; Line Search: 0.2672
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 10b2c1cf to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.671e+03, gradient 8.323e-03, dot -0.387; [0469b0c8-880e-4342-aeed-2cf2dcd17fcf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.527221648028044, -3.1170138608555464, -1.7980953180116086, -0.8013160573321375
LBFGS Accumulation History: 3 points
Removed measurement 10b2c1cf to history. Total: 3
th(0)=-4.527221648028044;dx=-6.589555891980652E-5
New Minimum: -4.527221648028044 > -4.892998269903028
END: th(108583.50837760697)=-4.892998269903028; dx=-2.870846921999546E-6 evalInputDelta=0.3657766218749847
Fitness changed from -4.527221648028044 to -4.892998269903028
Iteration 4 complete. Error: -4.892998269903028 Total: 1.1233; Orientation: 0.1856; Line Search: 0.7820
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 5a8835c6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.188e+03, gradient 9.532e-03, dot -0.210; [0469b0c8-880e-4342-aeed-2cf2dcd17fcf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.892998269903028, -3.1170138608555464, -1.7980953180116086, -0.8013160573321375
LBFGS Accumulation History: 3 points
Removed measurement 5a8835c6 to history. Total: 3
th(0)=-4.892998269903028;dx=-8.377729938237342E-5
New Minimum: -4.892998269903028 > -5.816907694528938
WOLF (strong): th(233936.07721408413)=-5.816907694528938; dx=3.855613985861103E-6 evalInputDelta=0.9239094246259096
New Minimum: -5.816907694528938 > -7.546637267656864
WOLF (strong): th(116968.03860704206)=-7.546637267656864; dx=2.3622564971677903E-6 evalInputDelta=2.653638997753836
New Minimum: -7.546637267656864 > -7.656039628584406
END: th(38989.346202347355)=-7.656039628584406; dx=-2.562662825536892E-5 evalInputDelta=2.7630413586813773
Fitness changed from -4.892998269903028 to -7.656039628584406
Iteration 5 complete. Error: -7.656039628584406 Total: 0.6247; Orientation: 0.0348; Line Search: 0.4578
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 6f627a1a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.332e+03, gradient 7.645e-03, dot -0.293; [0469b0c8-880e-4342-aeed-2cf2dcd17fcf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -7.656039628584406, -3.1170138608555464, -1.7980953180116086, -0.8013160573321375
LBFGS Accumulation History: 3 points
Removed measurement 6f627a1a to history. Total: 3
th(0)=-7.656039628584406;dx=-5.158977907367203E-5
New Minimum: -7.656039628584406 > -9.301744092385318
END: th(84000.00000000003)=-9.301744092385318; dx=-1.004303959123171E-5 evalInputDelta=1.6457044638009126
Fitness changed from -7.656039628584406 to -9.301744092385318
Iteration 6 complete. Error: -9.301744092385318 Total: 0.3895; Orientation: 0.0341; Line Search: 0.2310
Adding measurement 218c6d3e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.785e+03, gradient 7.512e-03, dot -0.340; [0469b0c8-880e-4342-aeed-2cf2dcd17fcf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -9.301744092385318, -3.1170138608555464, -1.7980953180116086, -0.8013160573321375
LBFGS Accumulation History: 3 points
Removed measurement 218c6d3e to history. Total: 3
th(0)=-9.301744092385318;dx=-5.081633938450253E-5
New Minimum: -9.301744092385318 > -9.584003894874542
WOLF (strong): th(180972.5139626783)=-9.584003894874542; dx=3.0855767998348856E-6 evalInputDelta=0.28225980248922333
New Minimum: -9.584003894874542 > -10.984681762660424
END: th(90486.25698133915)=-10.984681762660424; dx=-4.430145383702383E-6 evalInputDelta=1.6829376702751055
Fitness changed from -9.301744092385318 to -10.984681762660424
Iteration 7 complete. Error: -10.984681762660424 Total: 0.4797; Orientation: 0.0308; Line Search: 0.3392
Adding measurement 5680f009 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.454e+03, gradient 7.705e-03, dot -0.325; [0469b0c8-880e-4342-aeed-2cf2dcd17fcf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.984681762660424, -3.1170138608555464, -1.7980953180116086, -0.8013160573321375
LBFGS Accumulation History: 3 points
Removed measurement 5680f009 to history. Total: 3
th(0)=-10.984681762660424;dx=-5.427325692335053E-5
Armijo: th(194946.7310117368)=-9.436354814771159; dx=7.734244257066414E-6 evalInputDelta=-1.5483269478892652
New Minimum: -10.984681762660424 > -12.239430199269105
END: th(97473.3655058684)=-12.239430199269105; dx=-1.8271483976792187E-6 evalInputDelta=1.254748436608681
Fitness changed from -10.984681762660424 to -12.239430199269105
Iteration 8 complete. Error: -12.239430199269105 Total: 0.4800; Orientation: 0.0297; Line Search: 0.3388
Adding measurement 1606ecbb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.080e+04, gradient 7.743e-03, dot -0.344; [0469b0c8-880e-4342-aeed-2cf2dcd17fcf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.239430199269105, -3.1170138608555464, -1.7980953180116086, -0.8013160573321375
LBFGS Accumulation History: 3 points
Removed measurement 1606ecbb to history. Total: 3
th(0)=-12.239430199269105;dx=-5.689726923366621E-5
Armijo: th(210000.0000000001)=-11.616697084733941; dx=7.032720451867874E-6 evalInputDelta=-0.6227331145351638
New Minimum: -12.239430199269105 > -13.664973791154328
WOLF (strong): th(105000.00000000004)=-13.664973791154328; dx=1.5227521989950656E-6 evalInputDelta=1.4255435918852228
New Minimum: -13.664973791154328 > -13.69140890767039
END: th(35000.000000000015)=-13.69140890767039; dx=-2.1682128439471552E-5 evalInputDelta=1.4519787084012847
Fitness changed from -12.239430199269105 to -13.69140890767039
Iteration 9 complete. Error: -13.69140890767039 Total: 0.5884; Orientation: 0.0287; Line Search: 0.4517
Adding measurement 60e83cb9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.725e+04, gradient 6.105e-03, dot -0.452; [0469b0c8-880e-4342-aeed-2cf2dcd17fcf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -13.69140890767039, -3.1170138608555464, -1.7980953180116086, -0.8013160573321375
LBFGS Accumulation History: 3 points
Removed measurement 60e83cb9 to history. Total: 3
th(0)=-13.69140890767039;dx=-3.213662595367779E-5
New Minimum: -13.69140890767039 > -15.058384831623991
END: th(75405.21415111597)=-15.058384831623991; dx=-1.186510772417178E-5 evalInputDelta=1.3669759239536017
Fitness changed from -13.69140890767039 to -15.058384831623991
Iteration 10 complete. Error: -15.058384831623991 Total: 0.3669; Orientation: 0.0304; Line Search: 0.2278
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 58b2c0b4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.319e+04, gradient 6.185e-03, dot -0.428; [0469b0c8-880e-4342-aeed-2cf2dcd17fcf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -15.058384831623991, -3.1170138608555464, -1.7980953180116086, -0.8013160573321375
LBFGS Accumulation History: 3 points
Removed measurement 58b2c0b4 to history. Total: 3
th(0)=-15.058384831623991;dx=-3.323204643842649E-5
Armijo: th(162455.60917644735)=-15.010902402566535; dx=-1.533927681176921E-6 evalInputDelta=-0.0474824290574567
New Minimum: -15.058384831623991 > -15.872477363154218
END: th(81227.80458822368)=-15.872477363154218; dx=-5.74292143034506E-6 evalInputDelta=0.8140925315302265
Fitness changed from -15.058384831623991 to -15.872477363154218
Iteration 11 complete. Error: -15.872477363154218 Total: 0.8273; Orientation: 0.0284; Line Search: 0.6731
Adding measurement 5ea73927 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.481e+04, gradient 6.715e-03, dot -0.417; [0469b0c8-880e-4342-aeed-2cf2dcd17fcf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -15.872477363154218, -3.1170138608555464, -1.7980953180116086, -0.8013160573321375
LBFGS Accumulation History: 3 points
Removed measurement 5ea73927 to history. Total: 3
th(0)=-15.872477363154218;dx=-4.1771397323181765E-5
Armijo: th(175000.00000000012)=-15.106037085982948; dx=6.398352193015511E-6 evalInputDelta=-0.7664402771712702
New Minimum: -15.872477363154218 > -16.53053325603768
WOLF (strong): th(87500.00000000006)=-16.53053325603768; dx=5.667213204038145E-7 evalInputDelta=0.6580558928834606
New Minimum: -16.53053325603768 > -16.663563720041697
END: th(29166.666666666686)=-16.663563720041697; dx=-2.0726914549975506E-5 evalInputDelta=0.7910863568874795
Fitness changed from -15.872477363154218 to -16.663563720041697
Iteration 12 complete. Error: -16.663563720041697 Total: 0.5865; Orientation: 0.0289; Line Search: 0.4475
Adding measurement 77431f12 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.008e+04, gradient 5.292e-03, dot -0.516; [0469b0c8-880e-4342-aeed-2cf2dcd17fcf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.663563720041697, -3.1170138608555464, -1.7980953180116086, -0.8013160573321375
LBFGS Accumulation History: 3 points
Removed measurement 77431f12 to history. Total: 3
th(0)=-16.663563720041697;dx=-2.5364528463906278E-5
New Minimum: -16.663563720041697 > -17.34101457655997
END: th(62837.67845926332)=-17.34101457655997; dx=-1.203859039940755E-5 evalInputDelta=0.6774508565182735
Fitness changed from -16.663563720041697 to -17.34101457655997
Iteration 13 complete. Error: -17.34101457655997 Total: 0.3600; Orientation: 0.0274; Line Search: 0.2253
Adding measurement 48322f0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.833e+06, gradient 5.596e-03, dot -0.520; [0469b0c8-880e-4342-aeed-2cf2dcd17fcf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.34101457655997, -3.1170138608555464, -1.7980953180116086, -0.8013160573321375
LBFGS Accumulation History: 3 points
Removed measurement 48322f0 to history. Total: 3
th(0)=-17.34101457655997;dx=-2.8433284217189277E-5
New Minimum: -17.34101457655997 > -17.5383425869715
END: th(135379.67431370614)=-17.5383425869715; dx=-1.9773024872333033E-6 evalInputDelta=0.19732801041152825
Fitness changed from -17.34101457655997 to -17.5383425869715
Iteration 14 complete. Error: -17.5383425869715 Total: 0.3625; Orientation: 0.0285; Line Search: 0.2248
Adding measurement 35e4cf0c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.974e+04, gradient 7.198e-03, dot -0.363; [0469b0c8-880e-4342-aeed-2cf2dcd17fcf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.5383425869715, -3.1170138608555464, -1.7980953180116086, -0.8013160573321375
LBFGS Accumulation History: 3 points
Removed measurement 35e4cf0c to history. Total: 3
th(0)=-17.5383425869715;dx=-4.846414727532029E-5
Armijo: th(291666.66666666686)=-14.09816754905597; dx=9.22942106098694E-6 evalInputDelta=-3.4401750379155285
New Minimum: -17.5383425869715 > -17.64451730813719
WOLF (strong): th(145833.33333333343)=-17.64451730813719; dx=1.023613385026081E-5 evalInputDelta=0.10617472116569004
New Minimum: -17.64451730813719 > -18.899057918633225
END: th(48611.111111111146)=-18.899057918633225; dx=-1.445862536736347E-5 evalInputDelta=1.3607153316617264
Fitness changed from -17.5383425869715 to -18.899057918633225
Iteration 15 complete. Error: -18.899057918633225 Total: 0.5785; Orientation: 0.0275; Line Search: 0.4376
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -18.899057918633225 (> -Infinity) after 9.416s (< 5400.000s)
Returns
-18.899057918633225