BasicOptimizer.scala:89 executed in 9.77 seconds (0.151 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 162331418078500
Reset training subject: 162331549527900
Adding measurement 4a9bcfc1 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-19.48887982331407;dx=-0.0892764115714329
New Minimum: -19.48887982331407 > -19.773113391259198
WOLFE (weak): th(2.154434690031884)=-19.773113391259198; dx=-0.09010855898327877 evalInputDelta=0.28423356794512955
New Minimum: -19.773113391259198 > -20.05905663718581
WOLFE (weak): th(4.308869380063768)=-20.05905663718581; dx=-0.08946602255715255 evalInputDelta=0.5701768138717433
New Minimum: -20.05905663718581 > -21.217053532375864
WOLFE (weak): th(12.926608140191302)=-21.217053532375864; dx=-0.09416338450025863 evalInputDelta=1.728173709061796
New Minimum: -21.217053532375864 > -26.371522791743914
WOLFE (weak): th(51.70643256076521)=-26.371522791743914; dx=-0.09330541385148827 evalInputDelta=6.882642968429845
New Minimum: -26.371522791743914 > -32.9896223916988
END: th(258.53216280382605)=-32.9896223916988; dx=-0.007688956152483273 evalInputDelta=13.500742568384734
Fitness changed from -19.48887982331407 to -32.9896223916988
Iteration 1 complete. Error: -32.9896223916988 Total: 1.2322; Orientation: 0.0016; Line Search: 0.8693
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 19b11c43 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-32.9896223916988;dx=-0.09967698674337108
Armijo: th(556.9906600335335)=-12.593985657108131; dx=0.0012739969454681741 evalInputDelta=-20.39563673459067
New Minimum: -32.9896223916988 > -38.43061580882243
END: th(278.49533001676673)=-38.43061580882243; dx=-0.005248584178819125 evalInputDelta=5.440993417123629
Fitness changed from -32.9896223916988 to -38.43061580882243
Iteration 2 complete. Error: -38.43061580882243 Total: 0.4809; Orientation: 0.0015; Line Search: 0.3591
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 7663cc77 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-38.43061580882243;dx=-0.11489312389848826
Armijo: th(600.0)=-34.57183171096901; dx=-0.012625079200017923 evalInputDelta=-3.858784097853423
New Minimum: -38.43061580882243 > -48.20792115523521
END: th(300.0)=-48.20792115523521; dx=-0.018614320356233168 evalInputDelta=9.777305346412781
Fitness changed from -38.43061580882243 to -48.20792115523521
Iteration 3 complete. Error: -48.20792115523521 Total: 0.4642; Orientation: 0.0015; Line Search: 0.3434
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 333081e2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.910e+02, gradient 2.755e-01, dot -0.518; [86844fb5-9cf1-4c06-b460-2619113f1e51 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -48.20792115523521, -38.43061580882243, -32.9896223916988, -19.48887982331407
LBFGS Accumulation History: 3 points
Removed measurement 333081e2 to history. Total: 3
th(0)=-48.20792115523521;dx=-0.0666497490121642
New Minimum: -48.20792115523521 > -76.52511204459292
END: th(646.3304070095652)=-76.52511204459292; dx=-0.018292057573299228 evalInputDelta=28.317190889357704
Fitness changed from -48.20792115523521 to -76.52511204459292
Iteration 4 complete. Error: -76.52511204459292 Total: 0.3897; Orientation: 0.0266; Line Search: 0.2430
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 4b1b8aa to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.021e+02, gradient 2.778e-01, dot -0.399; [86844fb5-9cf1-4c06-b460-2619113f1e51 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -76.52511204459292, -38.43061580882243, -32.9896223916988, -19.48887982331407
LBFGS Accumulation History: 3 points
Removed measurement 4b1b8aa to history. Total: 3
th(0)=-76.52511204459292;dx=-0.07059254409865903
New Minimum: -76.52511204459292 > -97.43892818869931
END: th(1392.4766500838339)=-97.43892818869931; dx=-0.0019248266220388702 evalInputDelta=20.91381614410639
Fitness changed from -76.52511204459292 to -97.43892818869931
Iteration 5 complete. Error: -97.43892818869931 Total: 0.3739; Orientation: 0.0286; Line Search: 0.2241
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 29bb7290 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.874e+02, gradient 3.693e-01, dot -0.280; [86844fb5-9cf1-4c06-b460-2619113f1e51 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -97.43892818869931, -38.43061580882243, -32.9896223916988, -19.48887982331407
LBFGS Accumulation History: 3 points
Removed measurement 29bb7290 to history. Total: 3
th(0)=-97.43892818869931;dx=-0.1308834430364707
Armijo: th(3000.0000000000005)=-55.49986143352635; dx=0.005558745290011588 evalInputDelta=-41.93906675517296
New Minimum: -97.43892818869931 > -111.55656496818105
WOLF (strong): th(1500.0000000000002)=-111.55656496818105; dx=0.011698177979823367 evalInputDelta=14.117636779481742
New Minimum: -111.55656496818105 > -151.74518629103198
END: th(500.00000000000006)=-151.74518629103198; dx=-0.031637072075343234 evalInputDelta=54.30625810233268
Fitness changed from -97.43892818869931 to -151.74518629103198
Iteration 6 complete. Error: -151.74518629103198 Total: 0.5956; Orientation: 0.0265; Line Search: 0.4515
Adding measurement 2c135954 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.982e+03, gradient 2.800e-01, dot -0.374; [86844fb5-9cf1-4c06-b460-2619113f1e51 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -151.74518629103198, -38.43061580882243, -32.9896223916988, -19.48887982331407
LBFGS Accumulation History: 3 points
Removed measurement 2c135954 to history. Total: 3
th(0)=-151.74518629103198;dx=-0.07399015172117401
New Minimum: -151.74518629103198 > -202.99839323563623
END: th(1077.217345015942)=-202.99839323563623; dx=-0.02123371756686857 evalInputDelta=51.253206944604244
Fitness changed from -151.74518629103198 to -202.99839323563623
Iteration 7 complete. Error: -202.99839323563623 Total: 0.3741; Orientation: 0.0273; Line Search: 0.2332
Adding measurement 4df6efe0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.715e+04, gradient 3.561e-01, dot -0.330; [86844fb5-9cf1-4c06-b460-2619113f1e51 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -202.99839323563623, -38.43061580882243, -32.9896223916988, -19.48887982331407
LBFGS Accumulation History: 3 points
Removed measurement 4df6efe0 to history. Total: 3
th(0)=-202.99839323563623;dx=-0.1114567027532504
New Minimum: -202.99839323563623 > -225.7647160901145
END: th(2320.79441680639)=-225.7647160901145; dx=-0.0109598215623678 evalInputDelta=22.76632285447826
Fitness changed from -202.99839323563623 to -225.7647160901145
Iteration 8 complete. Error: -225.7647160901145 Total: 0.3646; Orientation: 0.0288; Line Search: 0.2238
Adding measurement 7214bafb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.549e+04, gradient 4.381e-01, dot -0.231; [86844fb5-9cf1-4c06-b460-2619113f1e51 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -225.7647160901145, -38.43061580882243, -32.9896223916988, -19.48887982331407
LBFGS Accumulation History: 3 points
Removed measurement 7214bafb to history. Total: 3
th(0)=-225.7647160901145;dx=-0.1766432430066731
New Minimum: -225.7647160901145 > -277.7198453477893
WOLF (strong): th(5000.000000000002)=-277.7198453477893; dx=0.007981273189602577 evalInputDelta=51.95512925767483
New Minimum: -277.7198453477893 > -414.02698432590114
END: th(2500.000000000001)=-414.02698432590114; dx=-0.008678123431065903 evalInputDelta=188.26226823578665
Fitness changed from -225.7647160901145 to -414.02698432590114
Iteration 9 complete. Error: -414.02698432590114 Total: 0.4921; Orientation: 0.0341; Line Search: 0.3463
Adding measurement 4382a303 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.396e+04, gradient 4.056e-01, dot -0.215; [86844fb5-9cf1-4c06-b460-2619113f1e51 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -414.02698432590114, -38.43061580882243, -32.9896223916988, -19.48887982331407
LBFGS Accumulation History: 3 points
Removed measurement 4382a303 to history. Total: 3
th(0)=-414.02698432590114;dx=-0.14892468306243795
Armijo: th(5386.086725079712)=-317.7046092677914; dx=0.011343229748892982 evalInputDelta=-96.32237505810974
New Minimum: -414.02698432590114 > -601.9687491888594
END: th(2693.043362539856)=-601.9687491888594; dx=-0.0023585864492060805 evalInputDelta=187.94176486295828
Fitness changed from -414.02698432590114 to -601.9687491888594
Iteration 10 complete. Error: -601.9687491888594 Total: 0.4744; Orientation: 0.0271; Line Search: 0.3357
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 60f8cb61 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.966e+03, gradient 4.353e-01, dot 0.329; [86844fb5-9cf1-4c06-b460-2619113f1e51 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-601.9687491888594;dx=-292.353176579394
New Minimum: -601.9687491888594 > -662.9094960852499
WOLF (strong): th(5801.986042015977)=-662.9094960852499; dx=0.05728095563819631 evalInputDelta=60.940746896390465
New Minimum: -662.9094960852499 > -663.3421534170733
WOLF (strong): th(2900.9930210079883)=-663.3421534170733; dx=0.1091349686785589 evalInputDelta=61.37340422821387
New Minimum: -663.3421534170733 > -667.9941990766088
WOLF (strong): th(966.9976736693294)=-667.9941990766088; dx=0.3447961254706803 evalInputDelta=66.02544988774935
New Minimum: -667.9941990766088 > -669.531295945156
WOLF (strong): th(241.74941841733235)=-669.531295945156; dx=1.2843327387935592 evalInputDelta=67.56254675629657
New Minimum: -669.531295945156 > -732.7503838279445
WOLF (strong): th(48.34988368346647)=-732.7503838279445; dx=5.340750744370174 evalInputDelta=130.78163463908504
New Minimum: -732.7503838279445 > -931.0954905378932
WOLF (strong): th(8.058313947244413)=-931.0954905378932; dx=11.900049644822422 evalInputDelta=329.12674134903375
New Minimum: -931.0954905378932 > -960.4656422798267
END: th(1.1511877067492018)=-960.4656422798267; dx=-114.35709148836699 evalInputDelta=358.4968930909673
Fitness changed from -601.9687491888594 to -960.4656422798267
Iteration 11 complete. Error: -960.4656422798267 Total: 1.0606; Orientation: 0.0268; Line Search: 0.9127
Adding measurement 3e988f76 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 6.234e+04, gradient 4.189e-01, dot -0.155; [86844fb5-9cf1-4c06-b460-2619113f1e51 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -960.4656422798267, -601.9687491888594, -38.43061580882243, -32.9896223916988, -19.48887982331407
Rejected: LBFGS Orientation magnitude: 6.017e+04, gradient 4.189e-01, dot -0.161; [86844fb5-9cf1-4c06-b460-2619113f1e51 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -960.4656422798267, -601.9687491888594, -38.43061580882243, -32.9896223916988
LBFGS Accumulation History: 3 points
Removed measurement 3e988f76 to history. Total: 4
Removed measurement 60f8cb61 to history. Total: 3
th(0)=-960.4656422798267;dx=-0.16864354688246846
New Minimum: -960.4656422798267 > -961.1195945354564
END: th(2.4801587301587316)=-961.1195945354564; dx=-0.1501537139365589 evalInputDelta=0.6539522556296333
Fitness changed from -960.4656422798267 to -961.1195945354564
Iteration 12 complete. Error: -961.1195945354564 Total: 0.4219; Orientation: 0.0579; Line Search: 0.2538
Adding measurement 278840d8 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.113e+04, gradient 4.157e-01, dot 0.191; [86844fb5-9cf1-4c06-b460-2619113f1e51 = 1.000/1.000e+00]
th(0)=-961.1195945354564;dx=-935.529053328302
Armijo: th(5.343340005039398)=-788.2254198514534; dx=41.841798212647205 evalInputDelta=-172.89417468400302
Armijo: th(2.671670002519699)=-833.8294190060699; dx=70.85847204551101 evalInputDelta=-127.29017552938649
New Minimum: -961.1195945354564 > -991.7369232815727
WOLF (strong): th(0.8905566675065663)=-991.7369232815727; dx=43.797149094172575 evalInputDelta=30.6173287461163
New Minimum: -991.7369232815727 > -1070.879078140945
END: th(0.22263916687664156)=-1070.879078140945; dx=-289.1528614944759 evalInputDelta=109.75948360548864
Fitness changed from -961.1195945354564 to -1070.879078140945
Iteration 13 complete. Error: -1070.879078140945 Total: 0.7055; Orientation: 0.0280; Line Search: 0.5693
Adding measurement 31c147dd to history. Total: 4
Accepted: LBFGS Orientation magnitude: 7.844e+03, gradient 4.047e-01, dot 0.209; [86844fb5-9cf1-4c06-b460-2619113f1e51 = 1.000/1.000e+00]
th(0)=-1070.879078140945;dx=-665.3812506298705
Armijo: th(0.47966154447883413)=-1070.0219039814092; dx=-25.404175723966258 evalInputDelta=-0.8571741595358162
New Minimum: -1070.879078140945 > -1083.0901393697573
END: th(0.23983077223941707)=-1083.0901393697573; dx=-92.78605934538088 evalInputDelta=12.211061228812241
Fitness changed from -1070.879078140945 to -1083.0901393697573
Iteration 14 complete. Error: -1083.0901393697573 Total: 0.4914; Orientation: 0.0337; Line Search: 0.3422
Adding measurement 4b89448f to history. Total: 5
Rejected: LBFGS Orientation magnitude: 1.328e+04, gradient 3.822e-01, dot -0.109; [86844fb5-9cf1-4c06-b460-2619113f1e51 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1083.0901393697573, -1070.879078140945, -961.1195945354564, -38.43061580882243, -32.9896223916988, -19.48887982331407
Rejected: LBFGS Orientation magnitude: 1.415e+04, gradient 3.822e-01, dot -0.119; [86844fb5-9cf1-4c06-b460-2619113f1e51 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1083.0901393697573, -1070.879078140945, -961.1195945354564, -38.43061580882243, -32.9896223916988
Accepted: LBFGS Orientation magnitude: 5.422e+04, gradient 3.822e-01, dot 0.165; [86844fb5-9cf1-4c06-b460-2619113f1e51 = 1.000/1.000e+00]
Overwriting history with 4 points
th(0)=-1083.0901393697573;dx=-3608.056593798217
Armijo: th(0.5166997354497358)=-773.9804180339448; dx=318.2212634671399 evalInputDelta=-309.1097213358124
Armijo: th(0.2583498677248679)=-848.8726347338569; dx=300.46030506494617 evalInputDelta=-234.21750463590035
Armijo: th(0.08611662257495596)=-991.8256525216516; dx=171.61779388032562 evalInputDelta=-91.26448684810566
Armijo: th(0.02152915564373899)=-1072.002040576416; dx=-328.6316418476434 evalInputDelta=-11.08809879334126
Armijo: th(0.004305831128747798)=-1082.1654217355126; dx=-445.75939093632064 evalInputDelta=-0.9247176342446437
Armijo: th(7.176385214579664E-4)=-1082.965546543379; dx=-353.5377147337383 evalInputDelta=-0.12459282637814795
Armijo: th(1.0251978877970948E-4)=-1083.0721928846665; dx=-493.94126532132964 evalInputDelta=-0.01794648509076069
Armijo: th(1.2814973597463684E-5)=-1083.087773451441; dx=-478.62499389211916 evalInputDelta=-0.0023659183161726105
Armijo: th(1.4238859552737426E-6)=-1083.08985084341; dx=-516.2488206135209 evalInputDelta=-2.8852634727627446E-4
Armijo: th(1.4238859552737426E-7)=-1083.0901393697573; dx=-428.5597988637235 evalInputDelta=0.0
Armijo: th(1.2944417775215842E-8)=-1083.090081664494; dx=-408.5278053067494 evalInputDelta=-5.7705263316165656E-5
Armijo: th(1.0787014812679868E-9)=-1083.0900239592274; dx=-530.496633542782 evalInputDelta=-1.1541052981556277E-4
MIN ALPHA (8.297703702061437E-11): th(0.0)=-1083.0901393697573
Fitness changed from -1083.0901393697573 to -1083.0901393697573
Static Iteration Total: 1.8465; Orientation: 0.1195; Line Search: 1.6169
Iteration 15 failed. Error: -1083.0901393697573
Previous Error: 0.0 -> -1083.0901393697573
Retrying iteration 15
Final threshold in iteration 15: -1083.0901393697573 (> -Infinity) after 9.768s (< 5400.000s)
Returns
-1083.0901393697573