BasicOptimizer.scala:89 executed in 10.16 seconds (0.336 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 452002502920800
Reset training subject: 452002698866200
Adding measurement 34f54f89 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-1.1086725698336724;dx=-9.70778979027461E-7
New Minimum: -1.1086725698336724 > -1.1086781080297192
WOLFE (weak): th(2.154434690031884)=-1.1086781080297192; dx=-9.624215082382547E-7 evalInputDelta=5.538196046872557E-6
New Minimum: -1.1086781080297192 > -1.1086835244804123
WOLFE (weak): th(4.308869380063768)=-1.1086835244804123; dx=-9.78009919145931E-7 evalInputDelta=1.0954646739946838E-5
New Minimum: -1.1086835244804123 > -1.1087057985949893
WOLFE (weak): th(12.926608140191302)=-1.1087057985949893; dx=-9.765639361793628E-7 evalInputDelta=3.3228761316950184E-5
New Minimum: -1.1087057985949893 > -1.1088056614605766
WOLFE (weak): th(51.70643256076521)=-1.1088056614605766; dx=-9.790867692446742E-7 evalInputDelta=1.3309162690422482E-4
New Minimum: -1.1088056614605766 > -1.1093385981681088
WOLFE (weak): th(258.53216280382605)=-1.1093385981681088; dx=-9.69529242017495E-7 evalInputDelta=6.660283344364082E-4
New Minimum: -1.1093385981681088 > -1.1126739782569932
WOLFE (weak): th(1551.1929768229563)=-1.1126739782569932; dx=-9.813962988810002E-7 evalInputDelta=0.0040014084233208624
New Minimum: -1.1126739782569932 > -1.1367110242799927
WOLFE (weak): th(10858.350837760694)=-1.1367110242799927; dx=-9.723994373057094E-7 evalInputDelta=0.02803845444632036
New Minimum: -1.1367110242799927 > -1.3339283929956285
WOLFE (weak): th(86866.80670208555)=-1.3339283929956285; dx=-9.706038623418186E-7 evalInputDelta=0.22525582316195614
New Minimum: -1.3339283929956285 > -2.4058005102956663
END: th(781801.26031877)=-2.4058005102956663; dx=-3.0323420639203936E-7 evalInputDelta=1.297127940461994
Fitness changed from -1.1086725698336724 to -2.4058005102956663
Iteration 1 complete. Error: -2.4058005102956663 Total: 1.9296; Orientation: 0.0022; Line Search: 1.4542
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement c5fe90a to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-2.4058005102956663;dx=-1.119545376790014E-6
New Minimum: -2.4058005102956663 > -3.6087765637725613
WOLF (strong): th(1684339.7559414052)=-3.6087765637725613; dx=4.938160744925403E-8 evalInputDelta=1.202976053476895
New Minimum: -3.6087765637725613 > -3.6523050526421565
END: th(842169.8779707026)=-3.6523050526421565; dx=-1.7163810962286982E-7 evalInputDelta=1.2465045423464902
Fitness changed from -2.4058005102956663 to -3.6523050526421565
Iteration 2 complete. Error: -3.6523050526421565 Total: 0.5629; Orientation: 0.0031; Line Search: 0.4138
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 24a5db6d to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-3.6523050526421565;dx=-9.95122277231297E-7
New Minimum: -3.6523050526421565 > -4.523977350037537
WOLF (strong): th(1814400.0)=-4.523977350037537; dx=1.1738476823447182E-7 evalInputDelta=0.8716722973953805
New Minimum: -4.523977350037537 > -4.757523202834071
END: th(907200.0)=-4.757523202834071; dx=-9.898358127736164E-8 evalInputDelta=1.1052181501919147
Fitness changed from -3.6523050526421565 to -4.757523202834071
Iteration 3 complete. Error: -4.757523202834071 Total: 0.5734; Orientation: 0.0027; Line Search: 0.4211
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 46ee9bff to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.421e+03, gradient 9.367e-04, dot -0.506; [0bd2024b-4e28-4eda-a168-d2fdda309404 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.757523202834071, -3.6523050526421565, -2.4058005102956663, -1.1086725698336724
LBFGS Accumulation History: 3 points
Removed measurement 46ee9bff to history. Total: 3
th(0)=-4.757523202834071;dx=-8.675868054973448E-7
New Minimum: -4.757523202834071 > -5.309501971238636
WOLF (strong): th(1954503.150796925)=-5.309501971238636; dx=1.552459996720257E-7 evalInputDelta=0.551978768404565
New Minimum: -5.309501971238636 > -5.694388828159249
END: th(977251.5753984625)=-5.694388828159249; dx=-3.8774771159719535E-8 evalInputDelta=0.9368656253251775
Fitness changed from -4.757523202834071 to -5.694388828159249
Iteration 4 complete. Error: -5.694388828159249 Total: 0.6063; Orientation: 0.0438; Line Search: 0.4146
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 1b6679c5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.064e+03, gradient 8.999e-04, dot -0.515; [0bd2024b-4e28-4eda-a168-d2fdda309404 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.694388828159249, -3.6523050526421565, -2.4058005102956663, -1.1086725698336724
LBFGS Accumulation History: 3 points
Removed measurement 1b6679c5 to history. Total: 3
th(0)=-5.694388828159249;dx=-7.974188962693456E-7
New Minimum: -5.694388828159249 > -6.176192514947338
WOLF (strong): th(2105424.6949267564)=-6.176192514947338; dx=1.861727405580707E-7 evalInputDelta=0.4818036867880888
New Minimum: -6.176192514947338 > -6.606456248788217
WOLF (strong): th(1052712.3474633782)=-6.606456248788217; dx=8.451100038595562E-9 evalInputDelta=0.9120674206289685
END: th(350904.1158211261)=-6.283319290479578; dx=-4.7352730888549444E-7 evalInputDelta=0.5889304623203291
Fitness changed from -5.694388828159249 to -6.606456248788217
Iteration 5 complete. Error: -6.606456248788217 Total: 0.7355; Orientation: 0.0401; Line Search: 0.5454
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 7718b69d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.874e+03, gradient 8.970e-04, dot -0.493; [0bd2024b-4e28-4eda-a168-d2fdda309404 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.606456248788217, -3.6523050526421565, -2.4058005102956663, -1.1086725698336724
LBFGS Accumulation History: 3 points
Removed measurement 7718b69d to history. Total: 3
th(0)=-6.606456248788217;dx=-7.997959621991803E-7
New Minimum: -6.606456248788217 > -7.467613828025124
END: th(756000.0)=-7.467613828025124; dx=-1.4676176129331492E-7 evalInputDelta=0.8611575792369068
Fitness changed from -6.606456248788217 to -7.467613828025124
Iteration 6 complete. Error: -7.467613828025124 Total: 0.4552; Orientation: 0.0401; Line Search: 0.2657
Adding measurement 67feb9e5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.093e+04, gradient 7.220e-04, dot -0.622; [0bd2024b-4e28-4eda-a168-d2fdda309404 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -7.467613828025124, -3.6523050526421565, -2.4058005102956663, -1.1086725698336724
LBFGS Accumulation History: 3 points
Removed measurement 67feb9e5 to history. Total: 3
th(0)=-7.467613828025124;dx=-5.169018002472411E-7
New Minimum: -7.467613828025124 > -8.357666573618877
END: th(1628752.6256641042)=-8.357666573618877; dx=-6.083729412866416E-8 evalInputDelta=0.8900527455937528
Fitness changed from -7.467613828025124 to -8.357666573618877
Iteration 7 complete. Error: -8.357666573618877 Total: 0.8048; Orientation: 0.0445; Line Search: 0.6257
Adding measurement 4fcd1e1f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.328e+04, gradient 9.212e-04, dot -0.487; [0bd2024b-4e28-4eda-a168-d2fdda309404 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.357666573618877, -3.6523050526421565, -2.4058005102956663, -1.1086725698336724
LBFGS Accumulation History: 3 points
Removed measurement 4fcd1e1f to history. Total: 3
th(0)=-8.357666573618877;dx=-8.427796379891199E-7
Armijo: th(3509041.1582112615)=-8.08068811909648; dx=2.228656327750655E-7 evalInputDelta=-0.27697845452239633
New Minimum: -8.357666573618877 > -9.253232179878584
WOLF (strong): th(1754520.5791056307)=-9.253232179878584; dx=1.724830322436032E-7 evalInputDelta=0.8955656062597068
New Minimum: -9.253232179878584 > -9.257579669256215
END: th(584840.1930352102)=-9.257579669256215; dx=-3.515888710746966E-7 evalInputDelta=0.8999130956373378
Fitness changed from -8.357666573618877 to -9.257579669256215
Iteration 8 complete. Error: -9.257579669256215 Total: 0.7171; Orientation: 0.0392; Line Search: 0.5296
Adding measurement 3d69ff97 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.792e+04, gradient 6.363e-04, dot -0.700; [0bd2024b-4e28-4eda-a168-d2fdda309404 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -9.257579669256215, -3.6523050526421565, -2.4058005102956663, -1.1086725698336724
LBFGS Accumulation History: 3 points
Removed measurement 3d69ff97 to history. Total: 3
th(0)=-9.257579669256215;dx=-4.0181510252979455E-7
New Minimum: -9.257579669256215 > -10.147790744202647
END: th(1260000.0000000002)=-10.147790744202647; dx=-2.158874806854363E-7 evalInputDelta=0.890211074946432
Fitness changed from -9.257579669256215 to -10.147790744202647
Iteration 9 complete. Error: -10.147790744202647 Total: 0.4462; Orientation: 0.0426; Line Search: 0.2692
Adding measurement 18680661 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.217e+04, gradient 6.865e-04, dot -0.642; [0bd2024b-4e28-4eda-a168-d2fdda309404 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.147790744202647, -3.6523050526421565, -2.4058005102956663, -1.1086725698336724
LBFGS Accumulation History: 3 points
Removed measurement 18680661 to history. Total: 3
th(0)=-10.147790744202647;dx=-4.6812303699092716E-7
New Minimum: -10.147790744202647 > -11.22519592847741
END: th(2714587.709440174)=-11.22519592847741; dx=-3.850708729805709E-8 evalInputDelta=1.0774051842747632
Fitness changed from -10.147790744202647 to -11.22519592847741
Iteration 10 complete. Error: -11.22519592847741 Total: 0.4511; Orientation: 0.0422; Line Search: 0.2757
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 37b51ef1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.043e+04, gradient 9.330e-04, dot -0.431; [0bd2024b-4e28-4eda-a168-d2fdda309404 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.22519592847741, -3.6523050526421565, -2.4058005102956663, -1.1086725698336724
LBFGS Accumulation History: 3 points
Removed measurement 37b51ef1 to history. Total: 3
th(0)=-11.22519592847741;dx=-8.605598680950964E-7
Armijo: th(5848401.930352103)=-9.720154601961784; dx=2.680629473182709E-7 evalInputDelta=-1.5050413265156255
New Minimum: -11.22519592847741 > -12.151304498139595
WOLF (strong): th(2924200.9651760515)=-12.151304498139595; dx=2.5488829584565137E-7 evalInputDelta=0.9261085696621851
New Minimum: -12.151304498139595 > -12.652046450276767
END: th(974733.6550586838)=-12.652046450276767; dx=-3.1580170180891206E-7 evalInputDelta=1.4268505217993575
Fitness changed from -11.22519592847741 to -12.652046450276767
Iteration 11 complete. Error: -12.652046450276767 Total: 0.7187; Orientation: 0.0420; Line Search: 0.5370
Adding measurement 9b320f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.774e+04, gradient 6.316e-04, dot -0.637; [0bd2024b-4e28-4eda-a168-d2fdda309404 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.652046450276767, -3.6523050526421565, -2.4058005102956663, -1.1086725698336724
LBFGS Accumulation History: 3 points
Removed measurement 9b320f to history. Total: 3
th(0)=-12.652046450276767;dx=-3.9641228392797913E-7
New Minimum: -12.652046450276767 > -13.857603089064616
END: th(2100000.0000000005)=-13.857603089064616; dx=-1.3320129529490684E-7 evalInputDelta=1.205556638787849
Fitness changed from -12.652046450276767 to -13.857603089064616
Iteration 12 complete. Error: -13.857603089064616 Total: 0.4350; Orientation: 0.0388; Line Search: 0.2654
Adding measurement e2fad11 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.763e+04, gradient 7.053e-04, dot -0.540; [0bd2024b-4e28-4eda-a168-d2fdda309404 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -13.857603089064616, -3.6523050526421565, -2.4058005102956663, -1.1086725698336724
LBFGS Accumulation History: 3 points
Removed measurement e2fad11 to history. Total: 3
th(0)=-13.857603089064616;dx=-4.915947377114899E-7
New Minimum: -13.857603089064616 > -14.27325170997392
WOLF (strong): th(4524312.849066957)=-14.27325170997392; dx=1.123374153041814E-7 evalInputDelta=0.41564862090930355
New Minimum: -14.27325170997392 > -14.78694230158318
WOLF (strong): th(2262156.4245334785)=-14.78694230158318; dx=8.362032931529201E-9 evalInputDelta=0.9293392125185633
END: th(754052.1415111595)=-14.521706940708864; dx=-2.444914637021592E-7 evalInputDelta=0.6641038516442475
Fitness changed from -13.857603089064616 to -14.78694230158318
Iteration 13 complete. Error: -14.78694230158318 Total: 0.7106; Orientation: 0.0408; Line Search: 0.5317
Adding measurement 60706f6d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.718e+04, gradient 7.939e-04, dot -0.454; [0bd2024b-4e28-4eda-a168-d2fdda309404 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -14.78694230158318, -3.6523050526421565, -2.4058005102956663, -1.1086725698336724
LBFGS Accumulation History: 3 points
Removed measurement 60706f6d to history. Total: 3
th(0)=-14.78694230158318;dx=-6.277083822137192E-7
New Minimum: -14.78694230158318 > -16.000221446574372
END: th(1624556.0917644731)=-16.000221446574372; dx=-2.2618302963425705E-8 evalInputDelta=1.2132791449911924
Fitness changed from -14.78694230158318 to -16.000221446574372
Iteration 14 complete. Error: -16.000221446574372 Total: 0.4338; Orientation: 0.0388; Line Search: 0.2692
Adding measurement b4f0371 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.681e+04, gradient 6.468e-04, dot -0.530; [0bd2024b-4e28-4eda-a168-d2fdda309404 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.000221446574372, -3.6523050526421565, -2.4058005102956663, -1.1086725698336724
LBFGS Accumulation History: 3 points
Removed measurement b4f0371 to history. Total: 3
th(0)=-16.000221446574372;dx=-4.151324755107284E-7
New Minimum: -16.000221446574372 > -16.571388941609793
WOLF (strong): th(3500000.0000000014)=-16.571388941609793; dx=9.65956403752742E-8 evalInputDelta=0.5711674950354215
New Minimum: -16.571388941609793 > -16.838090169936443
END: th(1750000.0000000007)=-16.838090169936443; dx=-4.764924620053195E-8 evalInputDelta=0.837868723362071
Fitness changed from -16.000221446574372 to -16.838090169936443
Iteration 15 complete. Error: -16.838090169936443 Total: 0.5704; Orientation: 0.0399; Line Search: 0.4029
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -16.838090169936443 (> -Infinity) after 10.162s (< 5400.000s)
Returns
-16.838090169936443