BasicOptimizer.scala:89 executed in 10.67 seconds (0.971 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 139353477332400
Reset training subject: 139353786896700
Adding measurement 99a8de3 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.2914879200712594;dx=-1.2603381192594767E-5
New Minimum: -0.2914879200712594 > -0.2915168507107407
WOLFE (weak): th(2.154434690031884)=-0.2915168507107407; dx=-1.2654213121890147E-5 evalInputDelta=2.8930639481283826E-5
New Minimum: -0.2915168507107407 > -0.29154552045526505
WOLFE (weak): th(4.308869380063768)=-0.29154552045526505; dx=-1.234211140671286E-5 evalInputDelta=5.7600384005651595E-5
New Minimum: -0.29154552045526505 > -0.2916601620399189
WOLFE (weak): th(12.926608140191302)=-0.2916601620399189; dx=-1.2489063360368356E-5 evalInputDelta=1.7224196865950336E-4
New Minimum: -0.2916601620399189 > -0.2921764258364328
WOLFE (weak): th(51.70643256076521)=-0.2921764258364328; dx=-1.2505193931168655E-5 evalInputDelta=6.88505765173375E-4
New Minimum: -0.2921764258364328 > -0.2949218273024073
WOLFE (weak): th(258.53216280382605)=-0.2949218273024073; dx=-1.2424143694379466E-5 evalInputDelta=0.0034339072311478924
New Minimum: -0.2949218273024073 > -0.3120131936396288
WOLFE (weak): th(1551.1929768229563)=-0.3120131936396288; dx=-1.266691580904993E-5 evalInputDelta=0.020525273568369418
New Minimum: -0.3120131936396288 > -0.43353806002502415
WOLFE (weak): th(10858.350837760694)=-0.43353806002502415; dx=-1.1618395761634776E-5 evalInputDelta=0.14205013995376475
New Minimum: -0.43353806002502415 > -0.8609956695898131
END: th(86866.80670208555)=-0.8609956695898131; dx=-2.8826478682328605E-6 evalInputDelta=0.5695077495185537
Fitness changed from -0.2914879200712594 to -0.8609956695898131
Iteration 1 complete. Error: -0.8609956695898131 Total: 1.9199; Orientation: 0.0191; Line Search: 1.2597
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 79135a38 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-0.8609956695898131;dx=-1.9373479813834835E-5
New Minimum: -0.8609956695898131 > -1.436017855830265
END: th(187148.86177126726)=-1.436017855830265; dx=-1.7513509225651962E-6 evalInputDelta=0.5750221862404519
Fitness changed from -0.8609956695898131 to -1.436017855830265
Iteration 2 complete. Error: -1.436017855830265 Total: 0.4376; Orientation: 0.0032; Line Search: 0.2718
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 31a024ea to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-1.436017855830265;dx=-1.671590134183E-5
New Minimum: -1.436017855830265 > -2.777596064422819
END: th(403200.00000000006)=-2.777596064422819; dx=-8.342828015195014E-7 evalInputDelta=1.341578208592554
Fitness changed from -1.436017855830265 to -2.777596064422819
Iteration 3 complete. Error: -2.777596064422819 Total: 0.4223; Orientation: 0.0016; Line Search: 0.2835
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 483fe83a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.030e+03, gradient 3.912e-03, dot -0.272; [81fc2831-fd72-4540-ba07-9e136d30bb3a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.777596064422819, -1.436017855830265, -0.8609956695898131, -0.2914879200712594
LBFGS Accumulation History: 3 points
Removed measurement 483fe83a to history. Total: 3
th(0)=-2.777596064422819;dx=-1.4236220234039146E-5
New Minimum: -2.777596064422819 > -4.742398397962939
END: th(868668.0670208557)=-4.742398397962939; dx=-6.799694416620514E-7 evalInputDelta=1.9648023335401197
Fitness changed from -2.777596064422819 to -4.742398397962939
Iteration 4 complete. Error: -4.742398397962939 Total: 0.5912; Orientation: 0.1912; Line Search: 0.2537
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 220e1b7b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.480e+03, gradient 3.941e-03, dot -0.261; [81fc2831-fd72-4540-ba07-9e136d30bb3a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.742398397962939, -1.436017855830265, -0.8609956695898131, -0.2914879200712594
LBFGS Accumulation History: 3 points
Removed measurement 220e1b7b to history. Total: 3
th(0)=-4.742398397962939;dx=-1.3778012601625356E-5
New Minimum: -4.742398397962939 > -6.19248462729408
WOLF (strong): th(1871488.617712673)=-6.19248462729408; dx=1.6124040394012855E-7 evalInputDelta=1.4500862293311414
New Minimum: -6.19248462729408 > -8.387796768362158
END: th(935744.3088563365)=-8.387796768362158; dx=-7.135015942025174E-7 evalInputDelta=3.645398370399219
Fitness changed from -4.742398397962939 to -8.387796768362158
Iteration 5 complete. Error: -8.387796768362158 Total: 0.8247; Orientation: 0.0431; Line Search: 0.3643
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 41962299 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.379e+04, gradient 4.657e-03, dot -0.286; [81fc2831-fd72-4540-ba07-9e136d30bb3a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.387796768362158, -1.436017855830265, -0.8609956695898131, -0.2914879200712594
LBFGS Accumulation History: 3 points
Removed measurement 41962299 to history. Total: 3
th(0)=-8.387796768362158;dx=-1.9199384804851402E-5
Armijo: th(2016000.0000000007)=-7.658038012487827; dx=1.084724977680631E-6 evalInputDelta=-0.7297587558743306
New Minimum: -8.387796768362158 > -11.09422691824622
WOLF (strong): th(1008000.0000000003)=-11.09422691824622; dx=1.2048820005866296E-6 evalInputDelta=2.7064301498840617
New Minimum: -11.09422691824622 > -12.9723213476699
END: th(336000.0000000001)=-12.9723213476699; dx=-5.4707584851346175E-6 evalInputDelta=4.584524579307743
Fitness changed from -8.387796768362158 to -12.9723213476699
Iteration 6 complete. Error: -12.9723213476699 Total: 0.6175; Orientation: 0.0310; Line Search: 0.4611
Adding measurement 6adc90e9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.978e+04, gradient 4.365e-03, dot -0.364; [81fc2831-fd72-4540-ba07-9e136d30bb3a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.9723213476699, -1.436017855830265, -0.8609956695898131, -0.2914879200712594
LBFGS Accumulation History: 3 points
Removed measurement 6adc90e9 to history. Total: 3
th(0)=-12.9723213476699;dx=-1.7698066387901953E-5
New Minimum: -12.9723213476699 > -16.39434683056645
END: th(723890.0558507132)=-16.39434683056645; dx=-1.8098291902443254E-6 evalInputDelta=3.422025482896551
Fitness changed from -12.9723213476699 to -16.39434683056645
Iteration 7 complete. Error: -16.39434683056645 Total: 0.3702; Orientation: 0.0327; Line Search: 0.2287
Adding measurement 67e7fe73 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.300e+07, gradient 5.583e-03, dot -0.340; [81fc2831-fd72-4540-ba07-9e136d30bb3a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.39434683056645, -1.436017855830265, -0.8609956695898131, -0.2914879200712594
LBFGS Accumulation History: 3 points
Removed measurement 67e7fe73 to history. Total: 3
th(0)=-16.39434683056645;dx=-2.818393589668694E-5
Armijo: th(1559573.8480938943)=-12.649896210515152; dx=2.3815139523043895E-6 evalInputDelta=-3.7444506200512997
New Minimum: -16.39434683056645 > -18.134847643304386
WOLF (strong): th(779786.9240469472)=-18.134847643304386; dx=1.2562551360712444E-6 evalInputDelta=1.7405008127379347
New Minimum: -18.134847643304386 > -20.68929658336961
END: th(259928.97468231572)=-20.68929658336961; dx=-7.697047957217032E-6 evalInputDelta=4.294949752803159
Fitness changed from -16.39434683056645 to -20.68929658336961
Iteration 8 complete. Error: -20.68929658336961 Total: 0.5920; Orientation: 0.0292; Line Search: 0.4523
Adding measurement 2c2f4769 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 3.488e+04, gradient 4.666e-03, dot 0.476; [81fc2831-fd72-4540-ba07-9e136d30bb3a = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-20.68929658336961;dx=-77.48019089769794
Armijo: th(560000.0000000002)=-18.698713341373324; dx=1.13432996583803E-5 evalInputDelta=-1.990583241996287
Armijo: th(280000.0000000001)=-18.700588842540608; dx=2.2788253770869172E-5 evalInputDelta=-1.9887077408290033
Armijo: th(93333.33333333337)=-18.701463855137; dx=7.173031887223548E-5 evalInputDelta=-1.9878327282326111
Armijo: th(23333.333333333343)=-18.701686279519542; dx=2.7476896151535214E-4 evalInputDelta=-1.9876103038500688
Armijo: th(4666.666666666669)=-18.70782073703023; dx=0.0014054765076960015 evalInputDelta=-1.9814758463393822
Armijo: th(777.7777777777782)=-18.688300800442278; dx=0.008454051518564825 evalInputDelta=-2.0009957829273333
Armijo: th(111.11111111111117)=-18.6564219557426; dx=0.057767984731122754 evalInputDelta=-2.0328746276270095
Armijo: th(13.888888888888896)=-18.881668990997273; dx=0.47260501640925495 evalInputDelta=-1.807627592372338
New Minimum: -20.68929658336961 > -21.118130056483995
WOLF (strong): th(1.5432098765432107)=-21.118130056483995; dx=2.4056803071606394 evalInputDelta=0.42883347311438413
New Minimum: -21.118130056483995 > -24.50051281777555
END: th(0.15432098765432106)=-24.50051281777555; dx=-17.12214799172355 evalInputDelta=3.81121623440594
Fitness changed from -20.68929658336961 to -24.50051281777555
Iteration 9 complete. Error: -24.50051281777555 Total: 1.3921; Orientation: 0.0303; Line Search: 1.2540
Adding measurement 23e782e0 to history. Total: 4
Accepted: LBFGS Orientation magnitude: 2.737e+04, gradient 4.970e-03, dot 0.341; [81fc2831-fd72-4540-ba07-9e136d30bb3a = 1.000/1.000e+00]
th(0)=-24.50051281777555;dx=-48.28556214288637
Armijo: th(0.3324744892024514)=-24.18676075325695; dx=-0.8509728349698382 evalInputDelta=-0.3137520645185994
New Minimum: -24.50051281777555 > -24.831220388593852
END: th(0.1662372446012257)=-24.831220388593852; dx=-4.5979642550434825 evalInputDelta=0.3307075708183014
Fitness changed from -24.50051281777555 to -24.831220388593852
Iteration 10 complete. Error: -24.831220388593852 Total: 0.5205; Orientation: 0.0373; Line Search: 0.3772
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 33e0d356 to history. Total: 5
Accepted: LBFGS Orientation magnitude: 2.291e+04, gradient 5.035e-03, dot 0.280; [81fc2831-fd72-4540-ba07-9e136d30bb3a = 1.000/1.000e+00]
th(0)=-24.831220388593852;dx=-34.491440648182504
Armijo: th(0.3581472865441961)=-23.678087032409856; dx=0.13714700659936768 evalInputDelta=-1.1531333561839965
Armijo: th(0.17907364327209804)=-24.232988819707924; dx=-1.1160919974203714 evalInputDelta=-0.5982315688859288
Armijo: th(0.05969121442403268)=-24.68906276436526; dx=-3.56207466425442 evalInputDelta=-0.14215762422859157
Armijo: th(0.01492280360600817)=-24.80253779084669; dx=-4.640618060465192 evalInputDelta=-0.02868259774716364
Armijo: th(0.0029845607212016343)=-24.825591486922857; dx=-4.74367031755918 evalInputDelta=-0.00562890167099539
Armijo: th(4.97426786866939E-4)=-24.830269326770882; dx=-4.829860282649058 evalInputDelta=-9.510618229704448E-4
Armijo: th(7.106096955241986E-5)=-24.831086550782043; dx=-5.175132085573194 evalInputDelta=-1.3383781180920096E-4
Armijo: th(8.882621194052483E-6)=-24.831203548114157; dx=-4.987332055492402 evalInputDelta=-1.6840479695190425E-5
Armijo: th(9.86957910450276E-7)=-24.831218615912316; dx=-5.226663263647273 evalInputDelta=-1.7726815357832493E-6
Armijo: th(9.869579104502759E-8)=-24.831220388593852; dx=-4.680498920250851 evalInputDelta=0.0
Armijo: th(8.972344640457055E-9)=-24.831220388593852; dx=-5.122958049728274 evalInputDelta=0.0
Armijo: th(7.476953867047545E-10)=-24.831218615912316; dx=-5.008823414539342 evalInputDelta=-1.7726815357832493E-6
MIN ALPHA (5.751502974651958E-11): th(0.0)=-24.831220388593852
Fitness changed from -24.831220388593852 to -24.831220388593852
Static Iteration Total: 2.5178; Orientation: 0.0419; Line Search: 2.3499
Iteration 11 failed. Error: -24.831220388593852
Previous Error: 0.0 -> -24.831220388593852
Retrying iteration 11
Reset training subject: 139363683429000
Adding measurement 33f909f to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-24.831220388593852;dx=-2.2732265755803206E-5
END: th(2.1544346900318838E-10)=-24.831220388593852; dx=-1.2108241705382332E-5 evalInputDelta=0.0
Fitness changed from -24.831220388593852 to -24.831220388593852
Static Iteration Total: 0.4454; Orientation: 0.0016; Line Search: 0.2247
Iteration 12 failed. Error: -24.831220388593852
Previous Error: 0.0 -> -24.831220388593852
Optimization terminated 12
Final threshold in iteration 12: -24.831220388593852 (> -Infinity) after 10.652s (< 5400.000s)
Returns
-24.831220388593852