BasicOptimizer.scala:88 executed in 948.88 seconds (17.624 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 1785384023117
Reset training subject: 1795708889652
Adding measurement 757377f0 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=7.272400468587875;dx=-9.8949679598765E-7
New Minimum: 7.272400468587875 > 7.272400006651878
WOLFE (weak): th(2.154434690031884)=7.272400006651878; dx=-9.889373213035772E-7 evalInputDelta=4.6193599700927734E-7
New Minimum: 7.272400006651878 > 7.272399067878723
WOLFE (weak): th(4.308869380063768)=7.272399067878723; dx=-9.889372091208747E-7 evalInputDelta=1.4007091522216797E-6
New Minimum: 7.272399067878723 > 7.272394344210625
WOLFE (weak): th(12.926608140191302)=7.272394344210625; dx=-9.889392344686205E-7 evalInputDelta=6.124377250671387E-6
New Minimum: 7.272394344210625 > 7.272374734282494
WOLFE (weak): th(51.70643256076521)=7.272374734282494; dx=-9.889279708105315E-7 evalInputDelta=2.5734305381774902E-5
New Minimum: 7.272374734282494 > 7.272272750735283
WOLFE (weak): th(258.53216280382605)=7.272272750735283; dx=-9.888460467795243E-7 evalInputDelta=1.2771785259246826E-4
New Minimum: 7.272272750735283 > 7.2716338485479355
WOLFE (weak): th(1551.1929768229563)=7.2716338485479355; dx=-9.884210936442375E-7 evalInputDelta=7.666200399398804E-4
New Minimum: 7.2716338485479355 > 7.2670412957668304
WOLFE (weak): th(10858.350837760694)=7.2670412957668304; dx=-9.853450560287706E-7 evalInputDelta=0.005359172821044922
New Minimum: 7.2670412957668304 > 7.23013012111187
WOLFE (weak): th(86866.80670208555)=7.23013012111187; dx=-9.576069776861578E-7 evalInputDelta=0.042270347476005554
New Minimum: 7.23013012111187 > 6.941128119826317
END: th(781801.26031877)=6.941128119826317; dx=-7.108095738254664E-7 evalInputDelta=0.33127234876155853
Fitness changed from 7.272400468587875 to 6.941128119826317
Iteration 1 complete. Error: 6.941128119826317 Total: 136.2111; Orientation: 0.1310; Line Search: 104.9330
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 70856964 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=6.941128119826317;dx=-6.385455358373589E-7
New Minimum: 6.941128119826317 > 6.4942425191402435
END: th(1684339.7559414052)=6.4942425191402435; dx=-4.3177680422435835E-7 evalInputDelta=0.4468856006860733
Fitness changed from 6.941128119826317 to 6.4942425191402435
Iteration 2 complete. Error: 6.4942425191402435 Total: 31.3765; Orientation: 0.1117; Line Search: 21.4550
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 4e897c5f to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=6.4942425191402435;dx=-4.177100612059357E-7
New Minimum: 6.4942425191402435 > 5.929793640971184
END: th(3628800.0)=5.929793640971184; dx=-2.2405996451587637E-7 evalInputDelta=0.5644488781690598
Fitness changed from 6.4942425191402435 to 5.929793640971184
Iteration 3 complete. Error: 5.929793640971184 Total: 31.6577; Orientation: 0.1184; Line Search: 21.1867
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 158fe74f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.236e+03, gradient 5.266e-04, dot -0.784; [1d3b1b32-544e-459e-9bf3-fdaa85cb586a = 1.000/1.000e+00]
Orientation rejected. Popping history element from 5.929793640971184, 6.4942425191402435, 6.941128119826317, 7.272400468587875
LBFGS Accumulation History: 3 points
Removed measurement 158fe74f to history. Total: 3
th(0)=5.929793640971184;dx=-2.7728669381128094E-7
New Minimum: 5.929793640971184 > 5.391488775610924
END: th(7818012.6031877)=5.391488775610924; dx=-5.2004964694396535E-8 evalInputDelta=0.53830486536026
Fitness changed from 5.929793640971184 to 5.391488775610924
Iteration 4 complete. Error: 5.391488775610924 Total: 32.6365; Orientation: 2.7334; Line Search: 20.1458
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 6a16fd7d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.314e+03, gradient 5.370e-04, dot -0.492; [1d3b1b32-544e-459e-9bf3-fdaa85cb586a = 1.000/1.000e+00]
Orientation rejected. Popping history element from 5.391488775610924, 6.4942425191402435, 6.941128119826317, 7.272400468587875
LBFGS Accumulation History: 3 points
Removed measurement 6a16fd7d to history. Total: 3
th(0)=5.391488775610924;dx=-2.883327030488467E-7
Armijo: th(1.684339755941405E7)=5.803031176328659; dx=1.5997676714615287E-7 evalInputDelta=-0.4115424007177353
New Minimum: 5.391488775610924 > 5.26169428229332
WOLF (strong): th(8421698.779707026)=5.26169428229332; dx=9.662788887199694E-8 evalInputDelta=0.12979449331760406
New Minimum: 5.26169428229332 > 5.161934852600098
END: th(2807232.9265690087)=5.161934852600098; dx=-6.036367665461389E-8 evalInputDelta=0.2295539230108261
Fitness changed from 5.391488775610924 to 5.161934852600098
Iteration 5 complete. Error: 5.161934852600098 Total: 54.2831; Orientation: 2.8291; Line Search: 41.0314
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 9d60c48 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.325e+03, gradient 4.210e-04, dot -0.574; [1d3b1b32-544e-459e-9bf3-fdaa85cb586a = 1.000/1.000e+00]
Orientation rejected. Popping history element from 5.161934852600098, 6.4942425191402435, 6.941128119826317, 7.272400468587875
LBFGS Accumulation History: 3 points
Removed measurement 9d60c48 to history. Total: 3
th(0)=5.161934852600098;dx=-1.7721891634547166E-7
New Minimum: 5.161934852600098 > 5.022929981350899
WOLF (strong): th(6048000.0)=5.022929981350899; dx=3.9816142825636236E-8 evalInputDelta=0.1390048712491989
New Minimum: 5.022929981350899 > 5.010536834597588
END: th(3024000.0)=5.010536834597588; dx=-3.290528028527093E-8 evalInputDelta=0.15139801800251007
Fitness changed from 5.161934852600098 to 5.010536834597588
Iteration 6 complete. Error: 5.010536834597588 Total: 45.1973; Orientation: 2.6642; Line Search: 32.0587
Adding measurement 49434bc2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.662e+03, gradient 4.154e-04, dot -0.570; [1d3b1b32-544e-459e-9bf3-fdaa85cb586a = 1.000/1.000e+00]
Orientation rejected. Popping history element from 5.010536834597588, 6.4942425191402435, 6.941128119826317, 7.272400468587875
LBFGS Accumulation History: 3 points
Removed measurement 49434bc2 to history. Total: 3
th(0)=5.010536834597588;dx=-1.72549639070107E-7
New Minimum: 5.010536834597588 > 4.9805114939808846
WOLF (strong): th(6515010.502656417)=4.9805114939808846; dx=6.36604935417363E-8 evalInputDelta=0.030025340616703033
New Minimum: 4.9805114939808846 > 4.908242553472519
WOLF (strong): th(3257505.2513282085)=4.908242553472519; dx=1.5134998785930405E-8 evalInputDelta=0.10229428112506866
END: th(1085835.0837760696)=4.940119870007038; dx=-8.876614937332629E-8 evalInputDelta=0.07041696459054947
Fitness changed from 5.010536834597588 to 4.908242553472519
Iteration 7 complete. Error: 4.908242553472519 Total: 54.8765; Orientation: 2.6701; Line Search: 42.1231
Adding measurement 32762a99 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.055e+03, gradient 4.632e-04, dot -0.500; [1d3b1b32-544e-459e-9bf3-fdaa85cb586a = 1.000/1.000e+00]
Orientation rejected. Popping history element from 4.908242553472519, 6.4942425191402435, 6.941128119826317, 7.272400468587875
LBFGS Accumulation History: 3 points
Removed measurement 32762a99 to history. Total: 3
th(0)=4.908242553472519;dx=-2.145341409198353E-7
New Minimum: 4.908242553472519 > 4.803894676268101
WOLF (strong): th(2339360.772140841)=4.803894676268101; dx=1.5751055342724675E-8 evalInputDelta=0.10434787720441818
END: th(1169680.3860704205)=4.822014160454273; dx=-8.325068345139068E-8 evalInputDelta=0.0862283930182457
Fitness changed from 4.908242553472519 to 4.803894676268101
Iteration 8 complete. Error: 4.803894676268101 Total: 43.5218; Orientation: 2.7691; Line Search: 30.4910
Adding measurement 5c3efbe6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.805e+03, gradient 3.690e-04, dot -0.565; [1d3
...skipping 2803 bytes...
lement from 4.59101890027523, 6.4942425191402435, 6.941128119826317, 7.272400468587875
LBFGS Accumulation History: 3 points
Removed measurement 4f4e6491 to history. Total: 3
th(0)=4.59101890027523;dx=-8.082515636103029E-8
New Minimum: 4.59101890027523 > 4.538922518491745
WOLF (strong): th(4200000.000000001)=4.538922518491745; dx=1.4580675017335706E-8 evalInputDelta=0.05209638178348541
END: th(2100000.0000000005)=4.5403056889772415; dx=-2.0369610608955773E-8 evalInputDelta=0.05071321129798889
Fitness changed from 4.59101890027523 to 4.538922518491745
Iteration 12 complete. Error: 4.538922518491745 Total: 43.1781; Orientation: 2.6957; Line Search: 30.1375
Adding measurement ea2b2d0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.793e+03, gradient 4.291e-04, dot -0.431; [1d3b1b32-544e-459e-9bf3-fdaa85cb586a = 1.000/1.000e+00]
Orientation rejected. Popping history element from 4.538922518491745, 6.4942425191402435, 6.941128119826317, 7.272400468587875
LBFGS Accumulation History: 3 points
Removed measurement ea2b2d0 to history. Total: 3
th(0)=4.538922518491745;dx=-1.6006625323906816E-7
Armijo: th(4524312.849066957)=4.571127757430077; dx=1.1275216481821899E-7 evalInputDelta=-0.032205238938331604
New Minimum: 4.538922518491745 > 4.47343935072422
WOLF (strong): th(2262156.4245334785)=4.47343935072422; dx=4.311042123093146E-8 evalInputDelta=0.06548316776752472
END: th(754052.1415111595)=4.486674979329109; dx=-9.287327253703021E-8 evalInputDelta=0.0522475391626358
Fitness changed from 4.538922518491745 to 4.47343935072422
Iteration 13 complete. Error: 4.47343935072422 Total: 55.3636; Orientation: 2.6405; Line Search: 42.6341
Adding measurement 13645ad0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.918e+03, gradient 3.432e-04, dot -0.485; [1d3b1b32-544e-459e-9bf3-fdaa85cb586a = 1.000/1.000e+00]
Orientation rejected. Popping history element from 4.47343935072422, 6.4942425191402435, 6.941128119826317, 7.272400468587875
LBFGS Accumulation History: 3 points
Removed measurement 13645ad0 to history. Total: 3
th(0)=4.47343935072422;dx=-1.177784338971968E-7
New Minimum: 4.47343935072422 > 4.4221856370568275
END: th(1624556.0917644731)=4.4221856370568275; dx=-1.6005592927359107E-8 evalInputDelta=0.05125371366739273
Fitness changed from 4.47343935072422 to 4.4221856370568275
Iteration 14 complete. Error: 4.4221856370568275 Total: 33.2985; Orientation: 2.8631; Line Search: 20.9108
Adding measurement 15a21246 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.859e+03, gradient 2.388e-04, dot -0.634; [1d3b1b32-544e-459e-9bf3-fdaa85cb586a = 1.000/1.000e+00]
Orientation rejected. Popping history element from 4.4221856370568275, 6.4942425191402435, 6.941128119826317, 7.272400468587875
LBFGS Accumulation History: 3 points
Removed measurement 15a21246 to history. Total: 3
th(0)=4.4221856370568275;dx=-5.703895703265037E-8
New Minimum: 4.4221856370568275 > 4.374707281589508
END: th(3500000.0000000014)=4.374707281589508; dx=-4.705823746997674E-9 evalInputDelta=0.04747835546731949
Fitness changed from 4.4221856370568275 to 4.374707281589508
Iteration 15 complete. Error: 4.374707281589508 Total: 33.7880; Orientation: 2.6970; Line Search: 20.6273
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Adding measurement 74f214fe to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.837e+03, gradient 3.202e-04, dot -0.477; [1d3b1b32-544e-459e-9bf3-fdaa85cb586a = 1.000/1.000e+00]
Orientation rejected. Popping history element from 4.374707281589508, 6.4942425191402435, 6.941128119826317, 7.272400468587875
LBFGS Accumulation History: 3 points
Removed measurement 74f214fe to history. Total: 3
th(0)=4.374707281589508;dx=-1.0253046084414506E-7
Armijo: th(7540521.415111597)=4.460198983550072; dx=5.69836397694066E-8 evalInputDelta=-0.08549170196056366
New Minimum: 4.374707281589508 > 4.363418988883495
WOLF (strong): th(3770260.7075557983)=4.363418988883495; dx=4.293210686935802E-8 evalInputDelta=0.011288292706012726
New Minimum: 4.363418988883495 > 4.337258040904999
END: th(1256753.569185266)=4.337258040904999; dx=-1.9821755682138835E-8 evalInputDelta=0.03744924068450928
Fitness changed from 4.374707281589508 to 4.337258040904999
Iteration 16 complete. Error: 4.337258040904999 Total: 55.6787; Orientation: 3.5851; Line Search: 42.7824
Adding measurement 4c9af42d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.814e+03, gradient 2.104e-04, dot -0.662; [1d3b1b32-544e-459e-9bf3-fdaa85cb586a = 1.000/1.000e+00]
Orientation rejected. Popping history element from 4.337258040904999, 6.4942425191402435, 6.941128119826317, 7.272400468587875
LBFGS Accumulation History: 3 points
Removed measurement 4c9af42d to history. Total: 3
th(0)=4.337258040904999;dx=-4.4277900802866197E-8
New Minimum: 4.337258040904999 > 4.29947904497385
END: th(2707593.4862741223)=4.29947904497385; dx=-1.544058476853362E-8 evalInputDelta=0.03777899593114853
Fitness changed from 4.337258040904999 to 4.29947904497385
Iteration 17 complete. Error: 4.29947904497385 Total: 33.5194; Orientation: 2.6915; Line Search: 21.0335
Adding measurement 178fb642 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.757e+03, gradient 2.411e-04, dot -0.567; [1d3b1b32-544e-459e-9bf3-fdaa85cb586a = 1.000/1.000e+00]
Orientation rejected. Popping history element from 4.29947904497385, 6.4942425191402435, 6.941128119826317, 7.272400468587875
LBFGS Accumulation History: 3 points
Removed measurement 178fb642 to history. Total: 3
th(0)=4.29947904497385;dx=-5.8152442913190924E-8
New Minimum: 4.29947904497385 > 4.29013704508543
WOLF (strong): th(5833333.333333336)=4.29013704508543; dx=1.7808440204701977E-8 evalInputDelta=0.009341999888420105
New Minimum: 4.29013704508543 > 4.271862111985683
WOLF (strong): th(2916666.666666668)=4.271862111985683; dx=5.9408911096582445E-9 evalInputDelta=0.02761693298816681
END: th(972222.2222222226)=4.27884703874588; dx=-2.725239184563828E-8 evalInputDelta=0.020632006227970123
Fitness changed from 4.29947904497385 to 4.271862111985683
Iteration 18 complete. Error: 4.271862111985683 Total: 53.7475; Orientation: 2.6314; Line Search: 41.8104
Adding measurement 1eee4319 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.773e+03, gradient 2.929e-04, dot -0.476; [1d3b1b32-544e-459e-9bf3-fdaa85cb586a = 1.000/1.000e+00]
Orientation rejected. Popping history element from 4.271862111985683, 6.4942425191402435, 6.941128119826317, 7.272400468587875
LBFGS Accumulation History: 3 points
Removed measurement 1eee4319 to history. Total: 3
th(0)=4.271862111985683;dx=-8.579877175454706E-8
New Minimum: 4.271862111985683 > 4.236509256064892
WOLF (strong): th(2094589.2819754435)=4.236509256064892; dx=1.6209617723836146E-8 evalInputDelta=0.035352855920791626
END: th(1047294.6409877217)=4.2405717968940735; dx=-3.3078717787962974E-8 evalInputDelta=0.031290315091609955
Fitness changed from 4.271862111985683 to 4.236509256064892
Iteration 19 complete. Error: 4.236509256064892 Total: 43.2038; Orientation: 2.7444; Line Search: 30.6115
Adding measurement 432c4276 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.665e+03, gradient 2.690e-04, dot -0.489; [1d3b1b32-544e-459e-9bf3-fdaa85cb586a = 1.000/1.000e+00]
Orientation rejected. Popping history element from 4.236509256064892, 6.4942425191402435, 6.941128119826317, 7.272400468587875
LBFGS Accumulation History: 3 points
Removed measurement 432c4276 to history. Total: 3
th(0)=4.236509256064892;dx=-7.238216719670886E-8
New Minimum: 4.236509256064892 > 4.209841877222061
WOLF (strong): th(2256327.9052284355)=4.209841877222061; dx=1.7024178947141678E-8 evalInputDelta=0.026667378842830658
END: th(1128163.9526142178)=4.21054121106863; dx=-2.1532441539959935E-8 evalInputDelta=0.025968044996261597
Fitness changed from 4.236509256064892 to 4.209841877222061
Iteration 20 complete. Error: 4.209841877222061 Total: 44.7898; Orientation: 2.7856; Line Search: 31.8663
<a id="p-17"></a>Iteration 20
<a id="p-16"></a>
Final threshold in iteration 21: 4.209841877222061 (> -Infinity) after 948.875s (< 5400.000s)
Returns
4.209841877222061