BasicOptimizer.scala:89 executed in 55.90 seconds (1.528 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 208781750892800
Reset training subject: 208782553929000
Adding measurement 123431a8 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=10.86923826338639;dx=-1.6673423301538235E-5
New Minimum: 10.86923826338639 > 10.869206465667224
WOLFE (weak): th(2.154434690031884)=10.869206465667224; dx=-1.6343175496139267E-5 evalInputDelta=3.1797719165282956E-5
New Minimum: 10.869206465667224 > 10.869173700284614
WOLFE (weak): th(4.308869380063768)=10.869173700284614; dx=-1.651607150786478E-5 evalInputDelta=6.456310177505031E-5
New Minimum: 10.869173700284614 > 10.869044203510311
WOLFE (weak): th(12.926608140191302)=10.869044203510311; dx=-1.6635149400449456E-5 evalInputDelta=1.940598760779011E-4
New Minimum: 10.869044203510311 > 10.868454789716901
WOLFE (weak): th(51.70643256076521)=10.868454789716901; dx=-1.7111941012524944E-5 evalInputDelta=7.83473669487833E-4
New Minimum: 10.868454789716901 > 10.86529216925282
WOLFE (weak): th(258.53216280382605)=10.86529216925282; dx=-1.6978756950587916E-5 evalInputDelta=0.003946094133569389
New Minimum: 10.86529216925282 > 10.84395030626449
WOLFE (weak): th(1551.1929768229563)=10.84395030626449; dx=-1.9318190447726163E-5 evalInputDelta=0.025287957121898685
New Minimum: 10.84395030626449 > 10.635142857480634
WOLFE (weak): th(10858.350837760694)=10.635142857480634; dx=-2.9896766497159224E-5 evalInputDelta=0.23409540590575517
New Minimum: 10.635142857480634 > 8.96209674567741
END: th(86866.80670208555)=8.96209674567741; dx=-1.4843210727863178E-5 evalInputDelta=1.907141517708979
Fitness changed from 10.86923826338639 to 8.96209674567741
Iteration 1 complete. Error: 8.96209674567741 Total: 9.0464; Orientation: 0.0155; Line Search: 6.8567
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 18f84035 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=8.96209674567741;dx=-9.607985380370073E-5
New Minimum: 8.96209674567741 > 8.230892214290979
END: th(187148.86177126726)=8.230892214290979; dx=-3.807489404845844E-6 evalInputDelta=0.7312045313864317
Fitness changed from 8.96209674567741 to 8.230892214290979
Iteration 2 complete. Error: 8.230892214290979 Total: 2.3364; Orientation: 0.0162; Line Search: 1.5720
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 4413fac0 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=8.230892214290979;dx=-1.3165828112592922E-4
New Minimum: 8.230892214290979 > 6.85367383377935
END: th(403200.00000000006)=6.85367383377935; dx=-3.241188791133622E-8 evalInputDelta=1.3772183805116285
Fitness changed from 8.230892214290979 to 6.85367383377935
Iteration 3 complete. Error: 6.85367383377935 Total: 2.4944; Orientation: 0.0160; Line Search: 1.7276
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 55806eef to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.245e+05, gradient 1.031e-02, dot -0.072; [ea5d2fe5-cb62-4291-9fb8-adf876aae82c = 1.000/1.000e+00]
Orientation rejected. Popping history element from 6.85367383377935, 8.230892214290979, 8.96209674567741, 10.86923826338639
LBFGS Accumulation History: 3 points
Removed measurement 55806eef to history. Total: 3
th(0)=6.85367383377935;dx=-1.0567943139527814E-4
New Minimum: 6.85367383377935 > 6.346917892019896
WOLF (strong): th(868668.0670208557)=6.346917892019896; dx=3.338374743600668E-6 evalInputDelta=0.5067559417594545
New Minimum: 6.346917892019896 > 0.6830903532866834
END: th(434334.03351042786)=0.6830903532866834; dx=-4.301783154048868E-6 evalInputDelta=6.170583480492667
Fitness changed from 6.85367383377935 to 0.6830903532866834
Iteration 4 complete. Error: 0.6830903532866834 Total: 3.2950; Orientation: 0.2976; Line Search: 2.2272
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 128fec7f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.183e+05, gradient 1.040e-02, dot -0.072; [ea5d2fe5-cb62-4291-9fb8-adf876aae82c = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.6830903532866834, 8.230892214290979, 8.96209674567741, 10.86923826338639
LBFGS Accumulation History: 3 points
Removed measurement 128fec7f to history. Total: 3
th(0)=0.6830903532866834;dx=-1.0786446296815036E-4
Armijo: th(935744.3088563365)=5.500299353694022; dx=4.884112635148438E-6 evalInputDelta=-4.817209000407338
New Minimum: 0.6830903532866834 > -4.178087120163159
END: th(467872.15442816826)=-4.178087120163159; dx=-2.8356444192059924E-6 evalInputDelta=4.8611774734498425
Fitness changed from 0.6830903532866834 to -4.178087120163159
Iteration 5 complete. Error: -4.178087120163159 Total: 3.3263; Orientation: 0.2979; Line Search: 2.2722
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 1c5f8681 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.094e+05, gradient 1.053e-02, dot -0.068; [ea5d2fe5-cb62-4291-9fb8-adf876aae82c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.178087120163159, 8.230892214290979, 8.96209674567741, 10.86923826338639
LBFGS Accumulation History: 3 points
Removed measurement 1c5f8681 to history. Total: 3
th(0)=-4.178087120163159;dx=-1.1056236385302967E-4
Armijo: th(1008000.0000000003)=3.1569526652332733; dx=6.645412304178027E-6 evalInputDelta=-7.335039785396432
New Minimum: -4.178087120163159 > -9.266723953808704
END: th(504000.0000000002)=-9.266723953808704; dx=-6.0565244469387305E-6 evalInputDelta=5.088636833645545
Fitness changed from -4.178087120163159 to -9.266723953808704
Iteration 6 complete. Error: -9.266723953808704 Total: 3.8719; Orientation: 0.2826; Line Search: 2.2940
Adding measurement 4b026cc5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.022e+04, gradient 1.062e-02, dot -0.054; [ea5d2fe5-cb62-4291-9fb8-adf876aae82c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -9.266723953808704, 8.230892214290979, 8.96209674567741, 10.86923826338639
LBFGS Accumulation History: 3 points
Removed measurement 4b026cc5 to history. Total: 3
th(0)=-9.266723953808704;dx=-1.1216198686821643E-4
Armijo: th(1085835.0837760698)=1.209861498018121; dx=7.768636674218355E-6 evalInputDelta=-10.476585451826825
New Minimum: -9.266723953808704 > -12.305762473910207
END: th(542917.5418880349)=-12.305762473910207; dx=-4.767248582827708E-6 evalInputDelta=3.039038520101503
Fitness changed from -9.266723953808704 to -12.305762473910207
Iteration 7 complete. Error: -12.305762473910207 Total: 3.3110; Orientation: 0.3171; Line Search: 2.3059
Adding measurement 3cdeb20f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.281e+04, gradient 1.103e-02, dot -0.048; [ea5d2fe5-cb62-4291-9fb8-adf876aae82c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.305762473910207, 8.230892214290979, 8.96209674567741, 10.86923826338639
LBFGS Accumulation History: 3 points
Removed measurement 3cdeb20f to history. Total: 3
th(0)=-12.305762473910207;dx=-1.2099853630766054E-4
Armijo: th(1169680.3860704207)=1.2162487763792633; dx=8.426666228131603E-6 evalInputDelta=-13.52201125028947
New Minimum: -12.305762473910207 > -14.978003686430512
END: th(584840.1930352104)=-14.978003686430512; dx=-3.971754315072903E-6 evalInputDelta=2.672241212520305
Fitness changed from -12.305762473910207 to -14.978003686430512
Iteration 8 complete. Error: -14.978003686430512 Total: 3.5217; Orientation: 0.2961; Line Search: 2.4791
Adding measurement 37e2c797 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.557e+04, gradient 1.130e-02, dot -0.045; [ea5d2fe5-cb62-4291-9fb8-adf876aae82c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -14.978003686430512, 8.230892214290979, 8.96209674567741, 10.86923826338639
LBFGS Accumulation History: 3 points
Removed measurement 37e2c797 to history. Total: 3
th(0)=-14.978003686430512;dx=-1.2731406638651785E-4
Armijo: th(1260000.0000000005)=0.1692389475879068; dx=7.77625749090649E-6 evalInputDelta=-15.147242634018419
New Minimum: -14.978003686430512 > -18.546349112761924
END: th(630000.0000000002)=-18.546349112761924; dx=-6.150043139204477E-6 evalInputDelta=3.568345426331412
Fitness changed from -14.978003686430512 to -18.546349112761924
Iteration 9 complete. Error: -18.546349112761924 Total: 3.2668; Orientation: 0.2890; Line Search: 2.2722
Adding measurement 1d36d8c0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.061e+04, gradient 1.175e-02, dot -0.041; [ea5d2fe5-cb62-4291-9fb8-adf876aae82c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.546349112761924, 8.230892214290979, 8.96209674567741, 10.86923826338639
LBFGS Accumulation History: 3 points
Removed measurement 1d36d8c0 to history. Total: 3
th(0)=-18.546349112761924;dx=-1.3765173147335688E-4
Armijo: th(1357293.8547200873)=5.579292118542217; dx=1.110418148660262E-5 evalInputDelta=-24.12564123130414
New Minimum: -18.546349112761924 > -19.5104399176155
END: th(678646.9273600436)=-19.5104399176155; dx=-4.276787463963616E-7 evalInputDelta=0.9640908048535763
Fitness changed from -18.546349112761924 to -19.5104399176155
Iteration 10 complete. Error: -19.5104399176155 Total: 3.6131; Orientation: 0.3095; Line Search: 2.5698
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 53aa5c2f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.754e+04, gradient 1.161e-02, dot -0.026; [ea5d2fe5-cb62-4291-9fb8-adf876aae82c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.5104399176155, 8.230892214290979, 8.96209674567741, 10.86923826338639
LBFGS Accumulation History: 3 points
Removed measurement 53aa5c2f to history. Total: 3
th(0)=-19.5104399176155;dx=-1.344742505053092E-4
Armijo: th(1462100.482588026)=6.338136255008028; dx=1.2350315224389841E-5 evalInputDelta=-25.848576172623527
New Minimum: -19.5104399176155 > -20.900700824397013
END: th(731050.241294013)=-20.900700824397013; dx=-2.485522682279748E-6 evalInputDelta=1.3902609067815135
Fitness changed from -19.5104399176155 to -20.900700824397013
Iteration 11 complete. Error: -20.900700824397013 Total: 3.3008; Orientation: 0.2879; Line Search: 2.2238
Adding measurement 6559225a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.843e+04, gradient 1.167e-02, dot -0.049; [ea5d2fe5-cb62-4291-9fb8-adf876aae82c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -20.900700824397013, 8.230892214290979, 8.96209674567741, 10.86923826338639
LBFGS Accumulation History: 3 points
Removed measurement 6559225a to history. Total: 3
th(0)=-20.900700824397013;dx=-1.3581671027565896E-4
Armijo: th(1575000.0000000007)=7.5815328685901875; dx=1.2828564398866808E-5 evalInputDelta=-28.4822336929872
New Minimum: -20.900700824397013 > -21.859745957174482
WOLF (strong): th(787500.0000000003)=-21.859745957174482; dx=1.877402540623513E-6 evalInputDelta=0.9590451327774687
New Minimum: -21.859745957174482 > -34.6749137688925
END: th(262500.0000000001)=-34.6749137688925; dx=-4.8146645230401446E-5 evalInputDelta=13.774212944495488
Fitness changed from -20.900700824397013 to -34.6749137688925
Iteration 12 complete. Error: -34.6749137688925 Total: 4.2789; Orientation: 0.3159; Line Search: 3.2192
Adding measurement 52e55065 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.073e+05, gradient 9.744e-03, dot -0.063; [ea5d2fe5-cb62-4291-9fb8-adf876aae82c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -34.6749137688925, 8.230892214290979, 8.96209674567741, 10.86923826338639
LBFGS Accumulation History: 3 points
Removed measurement 52e55065 to history. Total: 3
th(0)=-34.6749137688925;dx=-9.462553649532707E-5
New Minimum: -34.6749137688925 > -40.37951018826669
END: th(565539.1061333697)=-40.37951018826669; dx=-1.442253670038409E-5 evalInputDelta=5.704596419374191
Fitness changed from -34.6749137688925 to -40.37951018826669
Iteration 13 complete. Error: -40.37951018826669 Total: 2.6688; Orientation: 0.2816; Line Search: 1.4767
Adding measurement 76be8605 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.312e+05, gradient 1.081e-02, dot -0.059; [ea5d2fe5-cb62-4291-9fb8-adf876aae82c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -40.37951018826669, 8.230892214290979, 8.96209674567741, 10.86923826338639
LBFGS Accumulation History: 3 points
Removed measurement 76be8605 to history. Total: 3
th(0)=-40.37951018826669;dx=-1.1676435582570944E-4
Armijo: th(1218417.068823355)=-13.230013426134178; dx=1.2818511824920054E-5 evalInputDelta=-27.149496762132515
New Minimum: -40.37951018826669 > -41.30137744677356
END: th(609208.5344116775)=-41.30137744677356; dx=-3.659043519309081E-6 evalInputDelta=0.9218672585068646
Fitness changed from -40.37951018826669 to -41.30137744677356
Iteration 14 complete. Error: -41.30137744677356 Total: 3.2723; Orientation: 0.3164; Line Search: 2.2659
Adding measurement 1656c42a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.585e+05, gradient 1.146e-02, dot -0.055; [ea5d2fe5-cb62-4291-9fb8-adf876aae82c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -41.30137744677356, 8.230892214290979, 8.96209674567741, 10.86923826338639
LBFGS Accumulation History: 3 points
Removed measurement 1656c42a to history. Total: 3
th(0)=-41.30137744677356;dx=-1.305622400725819E-4
Armijo: th(1312500.0000000007)=-7.836441303537136; dx=1.652383016503786E-5 evalInputDelta=-33.46493614323642
Armijo: th(656250.0000000003)=-39.89725207802759; dx=1.0353511089677891E-7 evalInputDelta=-1.4041253687459658
New Minimum: -41.30137744677356 > -51.61811718320057
END: th(218750.00000000012)=-51.61811718320057; dx=-4.929155056504532E-5 evalInputDelta=10.31673973642701
Fitness changed from -41.30137744677356 to -51.61811718320057
Iteration 15 complete. Error: -51.61811718320057 Total: 4.2694; Orientation: 0.2861; Line Search: 3.1080
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -51.61811718320057 (> -Infinity) after 55.902s (< 5400.000s)
Returns
-51.61811718320057