BasicOptimizer.scala:89 executed in 56.48 seconds (1.696 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 209112199955300
Reset training subject: 209112989329800
Adding measurement 4ef1a6b6 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=8.862384115485376;dx=-1.640501414820872E-4
New Minimum: 8.862384115485376 > 8.862175380749475
WOLFE (weak): th(2.154434690031884)=8.862175380749475; dx=-1.6391097451687702E-4 evalInputDelta=2.0873473590121705E-4
New Minimum: 8.862175380749475 > 8.861967507146996
WOLFE (weak): th(4.308869380063768)=8.861967507146996; dx=-1.64178068938411E-4 evalInputDelta=4.166083383800867E-4
New Minimum: 8.861967507146996 > 8.861131257493348
WOLFE (weak): th(12.926608140191302)=8.861131257493348; dx=-1.639381699455596E-4 evalInputDelta=0.0012528579920285665
New Minimum: 8.861131257493348 > 8.857368266363304
WOLFE (weak): th(51.70643256076521)=8.857368266363304; dx=-1.643950982745987E-4 evalInputDelta=0.005015849122072424
New Minimum: 8.857368266363304 > 8.837221457663158
WOLFE (weak): th(258.53216280382605)=8.837221457663158; dx=-1.6454896681031267E-4 evalInputDelta=0.025162657822217938
New Minimum: 8.837221457663158 > 8.709032640832014
WOLFE (weak): th(1551.1929768229563)=8.709032640832014; dx=-1.6887073362591767E-4 evalInputDelta=0.15335147465336263
New Minimum: 8.709032640832014 > 7.727771599203487
WOLFE (weak): th(10858.350837760694)=7.727771599203487; dx=-1.6587344679310114E-4 evalInputDelta=1.134612516281889
New Minimum: 7.727771599203487 > 3.00928498688865
END: th(86866.80670208555)=3.00928498688865; dx=-4.7666227758015444E-5 evalInputDelta=5.853099128596726
Fitness changed from 8.862384115485376 to 3.00928498688865
Iteration 1 complete. Error: 3.00928498688865 Total: 9.2576; Orientation: 0.0153; Line Search: 7.0554
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 1fbdc053 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=3.00928498688865;dx=-1.6248004031797547E-4
New Minimum: 3.00928498688865 > -2.7598247382522487
END: th(187148.86177126726)=-2.7598247382522487; dx=-1.992188177510614E-5 evalInputDelta=5.769109725140899
Fitness changed from 3.00928498688865 to -2.7598247382522487
Iteration 2 complete. Error: -2.7598247382522487 Total: 2.2764; Orientation: 0.0158; Line Search: 1.5035
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 2c3cdb9d to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-2.7598247382522487;dx=-1.8037666661803816E-4
New Minimum: -2.7598247382522487 > -4.622753051320213
WOLF (strong): th(403200.00000000006)=-4.622753051320213; dx=2.6672261779228018E-6 evalInputDelta=1.8629283130679646
New Minimum: -4.622753051320213 > -9.470457277414113
END: th(201600.00000000003)=-9.470457277414113; dx=-2.3384925889801696E-5 evalInputDelta=6.710632539161864
Fitness changed from -2.7598247382522487 to -9.470457277414113
Iteration 3 complete. Error: -9.470457277414113 Total: 3.6074; Orientation: 0.0163; Line Search: 2.3029
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 19d9c440 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.762e+03, gradient 1.376e-02, dot -0.291; [8d130bb1-290a-49c8-b671-3d1889db0e2d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -9.470457277414113, -2.7598247382522487, 3.00928498688865, 8.862384115485376
LBFGS Accumulation History: 3 points
Removed measurement 19d9c440 to history. Total: 3
th(0)=-9.470457277414113;dx=-1.8933020092584392E-4
Armijo: th(434334.03351042786)=-9.296623399843494; dx=6.8312906792519244E-6 evalInputDelta=-0.17383387757061897
New Minimum: -9.470457277414113 > -16.38514105202259
END: th(217167.01675521393)=-16.38514105202259; dx=-1.991571107242501E-5 evalInputDelta=6.914683774608477
Fitness changed from -9.470457277414113 to -16.38514105202259
Iteration 4 complete. Error: -16.38514105202259 Total: 3.2888; Orientation: 0.2984; Line Search: 2.2296
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 5cff59c3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.733e+03, gradient 1.445e-02, dot -0.256; [8d130bb1-290a-49c8-b671-3d1889db0e2d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.38514105202259, -2.7598247382522487, 3.00928498688865, 8.862384115485376
LBFGS Accumulation History: 3 points
Removed measurement 5cff59c3 to history. Total: 3
th(0)=-16.38514105202259;dx=-2.087081730860815E-4
Armijo: th(467872.15442816826)=-13.641506630262864; dx=1.081307545334988E-5 evalInputDelta=-2.743634421759726
New Minimum: -16.38514105202259 > -23.011980719571973
END: th(233936.07721408413)=-23.011980719571973; dx=-1.3190205429353528E-5 evalInputDelta=6.626839667549383
Fitness changed from -16.38514105202259 to -23.011980719571973
Iteration 5 complete. Error: -23.011980719571973 Total: 3.6604; Orientation: 0.3367; Line Search: 2.5890
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 63a3534b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.242e+03, gradient 1.407e-02, dot -0.231; [8d130bb1-290a-49c8-b671-3d1889db0e2d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -23.011980719571973, -2.7598247382522487, 3.00928498688865, 8.862384115485376
LBFGS Accumulation History: 3 points
Removed measurement 63a3534b to history. Total: 3
th(0)=-23.011980719571973;dx=-1.9775462728729786E-4
Armijo: th(504000.0000000002)=-18.508729915281144; dx=1.2237927901129244E-5 evalInputDelta=-4.503250804290829
New Minimum: -23.011980719571973 > -29.389621018607677
END: th(252000.0000000001)=-29.389621018607677; dx=-1.6244088743547127E-5 evalInputDelta=6.377640299035704
Fitness changed from -23.011980719571973 to -29.389621018607677
Iteration 6 complete. Error: -29.389621018607677 Total: 3.3271; Orientation: 0.3024; Line Search: 2.2768
Adding measurement 2c5eedf7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.331e+03, gradient 1.404e-02, dot -0.208; [8d130bb1-290a-49c8-b671-3d1889db0e2d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -29.389621018607677, -2.7598247382522487, 3.00928498688865, 8.862384115485376
LBFGS Accumulation History: 3 points
Removed measurement 2c5eedf7 to history. Total: 3
th(0)=-29.389621018607677;dx=-1.9695808611899576E-4
Armijo: th(542917.5418880349)=-22.680060998486944; dx=1.612033382364848E-5 evalInputDelta=-6.709560020120733
New Minimum: -29.389621018607677 > -35.80326540199653
END: th(271458.77094401745)=-35.80326540199653; dx=-1.5328574179267218E-5 evalInputDelta=6.413644383388856
Fitness changed from -29.389621018607677 to -35.80326540199653
Iteration 7 complete. Error: -35.80326540199653 Total: 3.5331; Orientation: 0.3036; Line Search: 2.4653
Adding measurement 3f0c2ac2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.361e+03, gradient 1.379e-02, dot -0.190; [8d130bb1-290a-49c8-b671-3d1889db0e2d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -35.80326540199653, -2.7598247382522487, 3.00928498688865, 8.862384115485376
LBFGS Accumulation History: 3 points
Removed measurement 3f0c2ac2 to history. Total: 3
th(0)=-35.80326540199653;dx=-1.898890628173651E-4
Armijo: th(584840.1930352104)=-27.368119674689076; dx=1.6820099290680277E-5 evalInputDelta=-8.435145727307457
New Minimum: -35.80326540199653 > -42.22156917641951
END: th(292420.0965176052)=-42.22156917641951; dx=-1.5477471923794295E-5 evalInputDelta=6.418303774422974
Fitness changed from -35.80326540199653 to -42.22156917641951
Iteration 8 complete. Error: -42.22156917641951 Total: 3.4902; Orientation: 0.3208; Line Search: 2.5117
Adding measurement 56dedbd1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.223e+04, gradient 1.312e-02, dot -0.179; [8d130bb1-290a-49c8-b671-3d1889db0e2d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -42.22156917641951, -2.7598247382522487, 3.00928498688865, 8.862384115485376
LBFGS Accumulation History: 3 points
Removed measurement 56dedbd1 to history. Total: 3
th(0)=-42.22156917641951;dx=-1.7202623789818356E-4
Armijo: th(630000.0000000002)=-31.27299382988563; dx=1.622895275243195E-5 evalInputDelta=-10.948575346533879
New Minimum: -42.22156917641951 > -47.48400090965515
END: th(315000.0000000001)=-47.48400090965515; dx=-1.5440797033089272E-5 evalInputDelta=5.262431733235644
Fitness changed from -42.22156917641951 to -47.48400090965515
Iteration 9 complete. Error: -47.48400090965515 Total: 3.6163; Orientation: 0.3633; Line Search: 2.4745
Adding measurement 613883c0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.671e+04, gradient 1.288e-02, dot -0.169; [8d130bb1-290a-49c8-b671-3d1889db0e2d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -47.48400090965515, -2.7598247382522487, 3.00928498688865, 8.862384115485376
LBFGS Accumulation History: 3 points
Removed measurement 613883c0 to history. Total: 3
th(0)=-47.48400090965515;dx=-1.657287616532709E-4
Armijo: th(678646.9273600436)=-34.544095435369826; dx=1.689201146129857E-5 evalInputDelta=-12.939905474285325
New Minimum: -47.48400090965515 > -52.31940139668498
END: th(339323.4636800218)=-52.31940139668498; dx=-1.3655093222604688E-5 evalInputDelta=4.83540048702983
Fitness changed from -47.48400090965515 to -52.31940139668498
Iteration 10 complete. Error: -52.31940139668498 Total: 3.2687; Orientation: 0.3106; Line Search: 2.2307
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 4b4d5197 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.299e+04, gradient 1.284e-02, dot -0.160; [8d130bb1-290a-49c8-b671-3d1889db0e2d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -52.31940139668498, -2.7598247382522487, 3.00928498688865, 8.862384115485376
LBFGS Accumulation History: 3 points
Removed measurement 4b4d5197 to history. Total: 3
th(0)=-52.31940139668498;dx=-1.645634594028554E-4
Armijo: th(731050.241294013)=-37.56607860164449; dx=1.925702569978702E-5 evalInputDelta=-14.75332279504049
New Minimum: -52.31940139668498 > -57.041720334729064
END: th(365525.1206470065)=-57.041720334729064; dx=-1.3653668457991286E-5 evalInputDelta=4.722318938044083
Fitness changed from -52.31940139668498 to -57.041720334729064
Iteration 11 complete. Error: -57.041720334729064 Total: 3.6439; Orientation: 0.3066; Line Search: 2.6296
Adding measurement 321336ad to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.557e+04, gradient 1.271e-02, dot -0.155; [8d130bb1-290a-49c8-b671-3d1889db0e2d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -57.041720334729064, -2.7598247382522487, 3.00928498688865, 8.862384115485376
LBFGS Accumulation History: 3 points
Removed measurement 321336ad to history. Total: 3
th(0)=-57.041720334729064;dx=-1.6134764832916924E-4
Armijo: th(787500.0000000003)=-39.281643894298426; dx=2.0540149199427434E-5 evalInputDelta=-17.760076440430637
New Minimum: -57.041720334729064 > -60.79626025561471
END: th(393750.0000000002)=-60.79626025561471; dx=-9.96382281840476E-6 evalInputDelta=3.754539920885648
Fitness changed from -57.041720334729064 to -60.79626025561471
Iteration 12 complete. Error: -60.79626025561471 Total: 3.2248; Orientation: 0.3022; Line Search: 2.2150
Adding measurement 4ddfbbb2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.398e+04, gradient 1.266e-02, dot -0.141; [8d130bb1-290a-49c8-b671-3d1889db0e2d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -60.79626025561471, -2.7598247382522487, 3.00928498688865, 8.862384115485376
LBFGS Accumulation History: 3 points
Removed measurement 4ddfbbb2 to history. Total: 3
th(0)=-60.79626025561471;dx=-1.597204142920312E-4
Armijo: th(848308.6592000546)=-40.289252347694855; dx=2.1800146733281827E-5 evalInputDelta=-20.507007907919856
New Minimum: -60.79626025561471 > -64.25275241540709
END: th(424154.3296000273)=-64.25275241540709; dx=-8.107653915354173E-6 evalInputDelta=3.4564921597923757
Fitness changed from -60.79626025561471 to -64.25275241540709
Iteration 13 complete. Error: -64.25275241540709 Total: 3.4989; Orientation: 0.3032; Line Search: 2.5246
Adding measurement 11a802fd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.417e+04, gradient 1.260e-02, dot -0.138; [8d130bb1-290a-49c8-b671-3d1889db0e2d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -64.25275241540709, -2.7598247382522487, 3.00928498688865, 8.862384115485376
LBFGS Accumulation History: 3 points
Removed measurement 11a802fd to history. Total: 3
th(0)=-64.25275241540709;dx=-1.5876220392208563E-4
Armijo: th(913812.8016175163)=-41.270971888145084; dx=2.300029155786688E-5 evalInputDelta=-22.981780527262003
New Minimum: -64.25275241540709 > -67.43233062157468
END: th(456906.40080875816)=-67.43233062157468; dx=-6.829751761800023E-6 evalInputDelta=3.179578206167591
Fitness changed from -64.25275241540709 to -67.43233062157468
Iteration 14 complete. Error: -67.43233062157468 Total: 3.1966; Orientation: 0.2809; Line Search: 2.2470
Adding measurement 56824df1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.061e+05, gradient 1.246e-02, dot -0.128; [8d130bb1-290a-49c8-b671-3d1889db0e2d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -67.43233062157468, -2.7598247382522487, 3.00928498688865, 8.862384115485376
LBFGS Accumulation History: 3 points
Removed measurement 56824df1 to history. Total: 3
th(0)=-67.43233062157468;dx=-1.5516152203166541E-4
Armijo: th(984375.0000000006)=-40.75916413018027; dx=2.357113296305886E-5 evalInputDelta=-26.67316649139441
New Minimum: -67.43233062157468 > -69.93552710169104
END: th(492187.5000000003)=-69.93552710169104; dx=-5.614169338393835E-6 evalInputDelta=2.503196480116358
Fitness changed from -67.43233062157468 to -69.93552710169104
Iteration 15 complete. Error: -69.93552710169104 Total: 3.5636; Orientation: 0.3181; Line Search: 2.5231
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -69.93552710169104 (> -Infinity) after 56.475s (< 5400.000s)
Returns
-69.93552710169104