BasicOptimizer.scala:89 executed in 11.85 seconds (0.502 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 638953625649100
Reset training subject: 638953809681300
Adding measurement 588d6490 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-1.3994548891460803;dx=-1.519422923105821E-5
New Minimum: -1.3994548891460803 > -1.399519896579551
WOLFE (weak): th(2.154434690031884)=-1.399519896579551; dx=-1.515422431869068E-5 evalInputDelta=6.500743347070959E-5
New Minimum: -1.399519896579551 > -1.3995846044023486
WOLFE (weak): th(4.308869380063768)=-1.3995846044023486; dx=-1.518299365926473E-5 evalInputDelta=1.2971525626825908E-4
New Minimum: -1.3995846044023486 > -1.3998441471231944
WOLFE (weak): th(12.926608140191302)=-1.3998441471231944; dx=-1.5208344455673438E-5 evalInputDelta=3.8925797711408805E-4
New Minimum: -1.3998441471231944 > -1.401012457300848
WOLFE (weak): th(51.70643256076521)=-1.401012457300848; dx=-1.5207358421430304E-5 evalInputDelta=0.0015575681547677078
New Minimum: -1.401012457300848 > -1.4072562171327598
WOLFE (weak): th(258.53216280382605)=-1.4072562171327598; dx=-1.5283918408789126E-5 evalInputDelta=0.00780132798667954
New Minimum: -1.4072562171327598 > -1.4464908570643145
WOLFE (weak): th(1551.1929768229563)=-1.4464908570643145; dx=-1.5359090235793128E-5 evalInputDelta=0.04703596791823417
New Minimum: -1.4464908570643145 > -1.7407072055732582
WOLFE (weak): th(10858.350837760694)=-1.7407072055732582; dx=-1.6302040754575774E-5 evalInputDelta=0.3412523164271779
New Minimum: -1.7407072055732582 > -3.370429284108605
END: th(86866.80670208555)=-3.370429284108605; dx=-3.2446596527594123E-6 evalInputDelta=1.970974394962525
Fitness changed from -1.3994548891460803 to -3.370429284108605
Iteration 1 complete. Error: -3.370429284108605 Total: 1.9347; Orientation: 0.0022; Line Search: 1.4286
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 27eeaeb8 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-3.370429284108605;dx=-2.109801864205532E-5
New Minimum: -3.370429284108605 > -5.264454756320768
WOLF (strong): th(187148.86177126726)=-5.264454756320768; dx=2.537743033281648E-7 evalInputDelta=1.8940254722121632
END: th(93574.43088563363)=-5.150084530310087; dx=-1.0143357116918114E-6 evalInputDelta=1.7796552462014819
Fitness changed from -3.370429284108605 to -5.264454756320768
Iteration 2 complete. Error: -5.264454756320768 Total: 0.6369; Orientation: 0.0023; Line Search: 0.4692
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 5a9c9a89 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-5.264454756320768;dx=-2.0509775625191098E-5
New Minimum: -5.264454756320768 > -8.102932838069687
WOLF (strong): th(201600.00000000003)=-8.102932838069687; dx=1.558628247662807E-6 evalInputDelta=2.838478081748919
END: th(100800.00000000001)=-8.056675368336121; dx=-5.064344849409034E-6 evalInputDelta=2.7922206120153525
Fitness changed from -5.264454756320768 to -8.102932838069687
Iteration 3 complete. Error: -8.102932838069687 Total: 0.6431; Orientation: 0.0021; Line Search: 0.4655
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 493d4276 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.681e+03, gradient 3.915e-03, dot -0.407; [0e6e07dd-3cee-42ab-89ab-12d2c77fc084 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.102932838069687, -5.264454756320768, -3.370429284108605, -1.3994548891460803
LBFGS Accumulation History: 3 points
Removed measurement 493d4276 to history. Total: 3
th(0)=-8.102932838069687;dx=-1.5289088969918972E-5
New Minimum: -8.102932838069687 > -11.525652644079576
WOLF (strong): th(217167.01675521393)=-11.525652644079576; dx=6.721466239648122E-7 evalInputDelta=3.422719806009889
END: th(108583.50837760697)=-11.088773061882362; dx=-6.128858916527862E-6 evalInputDelta=2.9858402238126747
Fitness changed from -8.102932838069687 to -11.525652644079576
Iteration 4 complete. Error: -11.525652644079576 Total: 0.6768; Orientation: 0.0423; Line Search: 0.4626
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 5fb30473 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.895e+03, gradient 3.500e-03, dot -0.450; [0e6e07dd-3cee-42ab-89ab-12d2c77fc084 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.525652644079576, -5.264454756320768, -3.370429284108605, -1.3994548891460803
LBFGS Accumulation History: 3 points
Removed measurement 5fb30473 to history. Total: 3
th(0)=-11.525652644079576;dx=-1.2221767735876556E-5
New Minimum: -11.525652644079576 > -14.502133309136788
WOLF (strong): th(233936.07721408413)=-14.502133309136788; dx=6.92509619440327E-8 evalInputDelta=2.9764806650572115
END: th(116968.03860704206)=-14.075978386198654; dx=-5.233780836561193E-6 evalInputDelta=2.550325742119078
Fitness changed from -11.525652644079576 to -14.502133309136788
Iteration 5 complete. Error: -14.502133309136788 Total: 0.6724; Orientation: 0.0376; Line Search: 0.4672
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 5cb29e8a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.073e+03, gradient 3.350e-03, dot -0.502; [0e6e07dd-3cee-42ab-89ab-12d2c77fc084 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -14.502133309136788, -5.264454756320768, -3.370429284108605, -1.3994548891460803
LBFGS Accumulation History: 3 points
Removed measurement 5cb29e8a to history. Total: 3
th(0)=-14.502133309136788;dx=-1.1179729467691827E-5
New Minimum: -14.502133309136788 > -17.696403909061324
END: th(252000.0000000001)=-17.696403909061324; dx=-4.826222280032599E-7 evalInputDelta=3.194270599924536
Fitness changed from -14.502133309136788 to -17.696403909061324
Iteration 6 complete. Error: -17.696403909061324 Total: 0.5175; Orientation: 0.0409; Line Search: 0.3045
Adding measurement 413d24bf to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.021e+03, gradient 3.270e-03, dot -0.560; [0e6e07dd-3cee-42ab-89ab-12d2c77fc084 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.696403909061324, -5.264454756320768, -3.370429284108605, -1.3994548891460803
LBFGS Accumulation History: 3 points
Removed measurement 413d24bf to history. Total: 3
th(0)=-17.696403909061324;dx=-1.0672168627643132E-5
New Minimum: -17.696403909061324 > -20.285480116242695
WOLF (strong): th(542917.5418880349)=-20.285480116242695; dx=1.9624454571533567E-6 evalInputDelta=2.589076207181371
New Minimum: -20.285480116242695 > -21.231377974133565
END: th(271458.77094401745)=-21.231377974133565; dx=-8.093811918428954E-7 evalInputDelta=3.534974065072241
Fitness changed from -17.696403909061324 to -21.231377974133565
Iteration 7 complete. Error: -21.231377974133565 Total: 0.6530; Orientation: 0.0390; Line Search: 0.4583
Adding measurement 4d4d8dd0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.933e+03, gradient 3.163e-03, dot -0.560; [0e6e07dd-3cee-42ab-89ab-12d2c77fc084 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -21.231377974133565, -5.264454756320768, -3.370429284108605, -1.3994548891460803
LBFGS Accumulation History: 3 points
Removed measurement 4d4d8dd0 to history. Total: 3
th(0)=-21.231377974133565;dx=-9.963892596667477E-6
New Minimum: -21.231377974133565 > -23.931724554174814
WOLF (strong): th(584840.1930352104)=-23.931724554174814; dx=1.8336346188725089E-6 evalInputDelta=2.7003465800412485
New Minimum: -23.931724554174814 > -24.68534173411591
END: th(292420.0965176052)=-24.68534173411591; dx=-8.52935370338156E-7 evalInputDelta=3.4539637599823436
Fitness changed from -21.231377974133565 to -24.68534173411591
Iteration 8 complete. Error: -24.68534173411591 Total: 0.6669; Orientation: 0.0385; Line Search: 0.4759
Adding measurement 49f1abca to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.850e+04, gradient 3.167e-03, dot -0.591; [0e6e07dd-3cee-42ab-89ab-12d2c77fc084 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -24.68534173411591, -5.264454756320768, -3.370429284108605, -1.3994548891460803
LBFGS Accumulation History: 3 points
Removed measurement 49f1abca to history. Total: 3
th(0)=-24.68534173411591;dx=-1.0013114475903478E-5
New Minimum: -24.68534173411591 > -27.45780497375373
WOLF (strong): th(630000.0000000002)=-27.45780497375373; dx=1.7991277016520026E-6 evalInputDelta=2.7724632396378226
New Minimum: -27.45780497375373 > -28.289816815019574
END: th(315000.0000000001)=-28.289816815019574; dx=-6.14408555003206E-7 evalInputDelta=3.604475080903665
Fitness changed from -24.68534173411591 to -28.289816815019574
Iteration 9 complete. Error: -28.289816815019574 Total: 0.7188; Orientation: 0.0440; Line Search: 0.5252
Adding measurement 3b806940 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.558e+04, gradient 3.129e-03, dot -0.563; [0e6e07dd-3cee-42ab-89ab-12d2c77fc084 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -28.289816815019574, -5.264454756320768, -3.370429284108605, -1.3994548891460803
LBFGS Accumulation History: 3 points
Removed measurement 3b806940 to history. Total: 3
th(0)=-28.289816815019574;dx=-9.77170615775705E-6
New Minimum: -28.289816815019574 > -31.318701213584987
WOLF (strong): th(678646.9273600436)=-31.318701213584987; dx=1.7254958035554738E-6 evalInputDelta=3.0288843985654132
New Minimum: -31.318701213584987 > -32.14959117748892
END: th(339323.4636800218)=-32.14959117748892; dx=-7.036673533192227E-7 evalInputDelta=3.8597743624693486
Fitness changed from -28.289816815019574 to -32.14959117748892
Iteration 10 complete. Error: -32.14959117748892 Total: 0.7075; Orientation: 0.0407; Line Search: 0.4940
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 77690961 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.941e+04, gradient 3.060e-03, dot -0.601; [0e6e07dd-3cee-42ab-89ab-12d2c77fc084 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -32.14959117748892, -5.264454756320768, -3.370429284108605, -1.3994548891460803
LBFGS Accumulation History: 3 points
Removed measurement 77690961 to history. Total: 3
th(0)=-32.14959117748892;dx=-9.344838721226279E-6
New Minimum: -32.14959117748892 > -35.53560869276017
WOLF (strong): th(731050.241294013)=-35.53560869276017; dx=1.6411345843672065E-6 evalInputDelta=3.3860175152712486
New Minimum: -35.53560869276017 > -36.269749723852485
END: th(365525.1206470065)=-36.269749723852485; dx=-7.567236719948435E-7 evalInputDelta=4.120158546363562
Fitness changed from -32.14959117748892 to -36.269749723852485
Iteration 11 complete. Error: -36.269749723852485 Total: 0.8307; Orientation: 0.1879; Line Search: 0.4731
Adding measurement 4c5ad4be to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.376e+04, gradient 2.980e-03, dot -0.577; [0e6e07dd-3cee-42ab-89ab-12d2c77fc084 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -36.269749723852485, -5.264454756320768, -3.370429284108605, -1.3994548891460803
LBFGS Accumulation History: 3 points
Removed measurement 4c5ad4be to history. Total: 3
th(0)=-36.269749723852485;dx=-8.859558529293442E-6
New Minimum: -36.269749723852485 > -39.29761135264198
WOLF (strong): th(787500.0000000003)=-39.29761135264198; dx=1.7777943640312285E-6 evalInputDelta=3.027861628789495
New Minimum: -39.29761135264198 > -40.26040832089587
END: th(393750.0000000002)=-40.26040832089587; dx=-6.983599838511027E-7 evalInputDelta=3.990658597043385
Fitness changed from -36.269749723852485 to -40.26040832089587
Iteration 12 complete. Error: -40.26040832089587 Total: 0.6643; Orientation: 0.0384; Line Search: 0.4705
Adding measurement 1e08bbb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.309e+04, gradient 2.907e-03, dot -0.586; [0e6e07dd-3cee-42ab-89ab-12d2c77fc084 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -40.26040832089587, -5.264454756320768, -3.370429284108605, -1.3994548891460803
LBFGS Accumulation History: 3 points
Removed measurement 1e08bbb to history. Total: 3
th(0)=-40.26040832089587;dx=-8.42798563376231E-6
New Minimum: -40.26040832089587 > -43.40975610166249
WOLF (strong): th(848308.6592000546)=-43.40975610166249; dx=1.7731642170392497E-6 evalInputDelta=3.149347780766618
New Minimum: -43.40975610166249 > -44.45533503498198
END: th(424154.3296000273)=-44.45533503498198; dx=-6.482491731324382E-7 evalInputDelta=4.194926714086108
Fitness changed from -40.26040832089587 to -44.45533503498198
Iteration 13 complete. Error: -44.45533503498198 Total: 0.6524; Orientation: 0.0375; Line Search: 0.4585
Adding measurement 461047bb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.435e+04, gradient 2.765e-03, dot -0.557; [0e6e07dd-3cee-42ab-89ab-12d2c77fc084 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -44.45533503498198, -5.264454756320768, -3.370429284108605, -1.3994548891460803
LBFGS Accumulation History: 3 points
Removed measurement 461047bb to history. Total: 3
th(0)=-44.45533503498198;dx=-7.624112436436032E-6
New Minimum: -44.45533503498198 > -47.285886960591775
WOLF (strong): th(913812.8016175163)=-47.285886960591775; dx=1.7401868926384907E-6 evalInputDelta=2.8305519256097966
New Minimum: -47.285886960591775 > -48.47056853782778
END: th(456906.40080875816)=-48.47056853782778; dx=-5.672613453306132E-7 evalInputDelta=4.015233502845803
Fitness changed from -44.45533503498198 to -48.47056853782778
Iteration 14 complete. Error: -48.47056853782778 Total: 1.2086; Orientation: 0.5454; Line Search: 0.5150
Adding measurement 663cfa02 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.509e+04, gradient 2.632e-03, dot -0.549; [0e6e07dd-3cee-42ab-89ab-12d2c77fc084 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -48.47056853782778, -5.264454756320768, -3.370429284108605, -1.3994548891460803
LBFGS Accumulation History: 3 points
Removed measurement 663cfa02 to history. Total: 3
th(0)=-48.47056853782778;dx=-6.907152594109251E-6
New Minimum: -48.47056853782778 > -51.335365879944014
WOLF (strong): th(984375.0000000006)=-51.335365879944014; dx=1.678754543277526E-6 evalInputDelta=2.8647973421162334
New Minimum: -51.335365879944014 > -52.48526994797266
END: th(492187.5000000003)=-52.48526994797266; dx=-4.3831297526502237E-7 evalInputDelta=4.01470141014488
Fitness changed from -48.47056853782778 to -52.48526994797266
Iteration 15 complete. Error: -52.48526994797266 Total: 0.6530; Orientation: 0.0387; Line Search: 0.4623
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -52.48526994797266 (> -Infinity) after 11.851s (< 5400.000s)
Returns
-52.48526994797266