BasicOptimizer.scala:89 executed in 38.58 seconds (0.767 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 634998901522600
Reset training subject: 634999554307400
Adding measurement 3aa768c0 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-15.245563938357398;dx=-1.0447849683085625E-6
New Minimum: -15.245563938357398 > -15.24556765389268
WOLFE (weak): th(2.154434690031884)=-15.24556765389268; dx=-1.0436677731013373E-6 evalInputDelta=3.7155352821116594E-6
New Minimum: -15.24556765389268 > -15.2455701309157
WOLFE (weak): th(4.308869380063768)=-15.2455701309157; dx=-1.0447118613501563E-6 evalInputDelta=6.192558302586804E-6
New Minimum: -15.2455701309157 > -15.245583135279947
WOLFE (weak): th(12.926608140191302)=-15.245583135279947; dx=-1.0440344683959995E-6 evalInputDelta=1.9196922549369333E-5
New Minimum: -15.245583135279947 > -15.245640106649711
WOLFE (weak): th(51.70643256076521)=-15.245640106649711; dx=-1.0438825265616646E-6 evalInputDelta=7.616829231338329E-5
New Minimum: -15.245640106649711 > -15.245944156773122
WOLFE (weak): th(258.53216280382605)=-15.245944156773122; dx=-1.04549047500828E-6 evalInputDelta=3.802184157244426E-4
New Minimum: -15.245944156773122 > -15.247842629982095
WOLFE (weak): th(1551.1929768229563)=-15.247842629982095; dx=-1.0442688530403455E-6 evalInputDelta=0.0022786916246975153
New Minimum: -15.247842629982095 > -15.261494644089444
WOLFE (weak): th(10858.350837760694)=-15.261494644089444; dx=-1.0416037112692878E-6 evalInputDelta=0.015930705732046135
New Minimum: -15.261494644089444 > -15.372490834474666
WOLFE (weak): th(86866.80670208555)=-15.372490834474666; dx=-1.0263372800004006E-6 evalInputDelta=0.12692689611726848
New Minimum: -15.372490834474666 > -16.284182180028186
END: th(781801.26031877)=-16.284182180028186; dx=-8.686920016685224E-7 evalInputDelta=1.0386182416707879
Fitness changed from -15.245563938357398 to -16.284182180028186
Iteration 1 complete. Error: -16.284182180028186 Total: 7.7830; Orientation: 0.0124; Line Search: 5.8588
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 3d1d3525 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-16.284182180028186;dx=-9.476931628634775E-7
New Minimum: -16.284182180028186 > -17.87728018004986
END: th(1684339.7559414052)=-17.87728018004986; dx=-6.475072220430228E-7 evalInputDelta=1.5930980000216728
Fitness changed from -16.284182180028186 to -17.87728018004986
Iteration 2 complete. Error: -17.87728018004986 Total: 1.8496; Orientation: 0.0075; Line Search: 1.1462
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 1c0ed7c1 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-17.87728018004986;dx=-8.13450768265911E-7
New Minimum: -17.87728018004986 > -20.36030914496444
END: th(3628800.0)=-20.36030914496444; dx=-4.7648092762694876E-7 evalInputDelta=2.483028964914581
Fitness changed from -17.87728018004986 to -20.36030914496444
Iteration 3 complete. Error: -20.36030914496444 Total: 1.7438; Orientation: 0.0074; Line Search: 1.1231
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 6cb62e90 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.255e+04, gradient 8.614e-04, dot -0.569; [f179b27f-ca36-438a-b310-7ef413dfb443 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -20.36030914496444, -17.87728018004986, -16.284182180028186, -15.245563938357398
LBFGS Accumulation History: 3 points
Removed measurement 6cb62e90 to history. Total: 3
th(0)=-20.36030914496444;dx=-7.366751299937443E-7
New Minimum: -20.36030914496444 > -25.05457691965292
END: th(7818012.6031877)=-25.05457691965292; dx=-3.3882273373700583E-7 evalInputDelta=4.69426777468848
Fitness changed from -20.36030914496444 to -25.05457691965292
Iteration 4 complete. Error: -25.05457691965292 Total: 2.1910; Orientation: 0.1678; Line Search: 1.2625
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 4c2f971 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.498e+04, gradient 8.065e-04, dot -0.480; [f179b27f-ca36-438a-b310-7ef413dfb443 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -25.05457691965292, -17.87728018004986, -16.284182180028186, -15.245563938357398
LBFGS Accumulation History: 3 points
Removed measurement 4c2f971 to history. Total: 3
th(0)=-25.05457691965292;dx=-6.451774138603778E-7
New Minimum: -25.05457691965292 > -33.87587654385407
END: th(1.684339755941405E7)=-33.87587654385407; dx=-1.7436012231098968E-7 evalInputDelta=8.82129962420115
Fitness changed from -25.05457691965292 to -33.87587654385407
Iteration 5 complete. Error: -33.87587654385407 Total: 1.9082; Orientation: 0.1422; Line Search: 1.1668
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 7407d953 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.683e+04, gradient 7.002e-04, dot -0.362; [f179b27f-ca36-438a-b310-7ef413dfb443 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -33.87587654385407, -17.87728018004986, -16.284182180028186, -15.245563938357398
LBFGS Accumulation History: 3 points
Removed measurement 7407d953 to history. Total: 3
th(0)=-33.87587654385407;dx=-4.878635464067017E-7
New Minimum: -33.87587654385407 > -43.51595222511602
END: th(3.6288E7)=-43.51595222511602; dx=-4.9957233434579346E-8 evalInputDelta=9.640075681261948
Fitness changed from -33.87587654385407 to -43.51595222511602
Iteration 6 complete. Error: -43.51595222511602 Total: 1.9421; Orientation: 0.1478; Line Search: 1.1300
Adding measurement 73128b3a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.438e+04, gradient 5.721e-04, dot -0.225; [f179b27f-ca36-438a-b310-7ef413dfb443 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -43.51595222511602, -17.87728018004986, -16.284182180028186, -15.245563938357398
LBFGS Accumulation History: 3 points
Removed measurement 73128b3a to history. Total: 3
th(0)=-43.51595222511602;dx=-3.248850658127839E-7
New Minimum: -43.51595222511602 > -52.22143942626336
END: th(7.8180126031877E7)=-52.22143942626336; dx=-3.3131627570822635E-9 evalInputDelta=8.705487201147342
Fitness changed from -43.51595222511602 to -52.22143942626336
Iteration 7 complete. Error: -52.22143942626336 Total: 2.0576; Orientation: 0.1599; Line Search: 1.3151
Adding measurement 7bce4a64 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.196e+04, gradient 4.149e-04, dot -0.121; [f179b27f-ca36-438a-b310-7ef413dfb443 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -52.22143942626336, -17.87728018004986, -16.284182180028186, -15.245563938357398
LBFGS Accumulation History: 3 points
Removed measurement 7bce4a64 to history. Total: 3
th(0)=-52.22143942626336;dx=-1.7041592593251604E-7
New Minimum: -52.22143942626336 > -62.158999289118356
WOLF (strong): th(1.6843397559414053E8)=-62.158999289118356; dx=2.0354985761341753E-9 evalInputDelta=9.937559862854997
New Minimum: -62.158999289118356 > -66.36811867093765
END: th(8.421698779707026E7)=-66.36811867093765; dx=-1.1729530154470542E-8 evalInputDelta=14.146679244674289
Fitness changed from -52.22143942626336 to -66.36811867093765
Iteration 8 complete. Error: -66.36811867093765 Total: 2.3458; Orientation: 0.1477; Line Search: 1.6462
Adding measurement 38ae6655 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.033e+04, gradient 2.996e-04, dot -0.116; [f179b27f-ca36-438a-b310-7ef413dfb443 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -66.36811867093765, -17.87728018004986, -16.284182180028186, -15.245563938357398
LBFGS Accumulation History: 3 points
Removed measurement 38ae6655 to history. Total: 3
th(0)=-66.36811867093765;dx=-8.882302898590241E-8
New Minimum: -66.36811867093765 > -69.45492419069193
WOLF (strong): th(1.8144000000000003E8)=-69.45492419069193; dx=3.750648164330628E-10 evalInputDelta=3.0868055197542787
New Minimum: -69.45492419069193 > -72.70265133645402
END: th(9.072000000000001E7)=-72.70265133645402; dx=-1.1204247586321839E-8 evalInputDelta=6.334532665516377
Fitness changed from -66.36811867093765 to -72.70265133645402
Iteration 9 complete. Error: -72.70265133645402 Total: 2.4990; Orientation: 0.1382; Line Search: 1.8268
Adding measurement 56122c1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.075e+04, gradient 2.431e-04, dot -0.124; [f179b27f-ca36-438a-b310-7ef413dfb443 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -72.70265133645402, -17.87728018004986, -16.284182180028186, -15.245563938357398
LBFGS Accumulation History: 3 points
Removed measurement 56122c1 to history. Total: 3
th(0)=-72.70265133645402;dx=-5.7710462116441567E-8
New Minimum: -72.70265133645402 > -74.73275430364612
END: th(1.9545031507969254E8)=-74.73275430364612; dx=-1.726324345330085E-10 evalInputDelta=2.030102967192093
Fitness changed from -72.70265133645402 to -74.73275430364612
Iteration 10 complete. Error: -74.73275430364612 Total: 1.7966; Orientation: 0.1405; Line Search: 1.0967
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 33ce2d24 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.254e+04, gradient 2.305e-04, dot -0.113; [f179b27f-ca36-438a-b310-7ef413dfb443 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -74.73275430364612, -17.87728018004986, -16.284182180028186, -15.245563938357398
LBFGS Accumulation History: 3 points
Removed measurement 33ce2d24 to history. Total: 3
th(0)=-74.73275430364612;dx=-5.233428530625429E-8
Armijo: th(4.2108493898535144E8)=-72.07911833372809; dx=3.0714549939197693E-9 evalInputDelta=-2.653635969918028
New Minimum: -74.73275430364612 > -77.67716594443557
WOLF (strong): th(2.1054246949267572E8)=-77.67716594443557; dx=5.18803990294056E-10 evalInputDelta=2.944411640789454
New Minimum: -77.67716594443557 > -79.08398984704732
END: th(7.018082316422524E7)=-79.08398984704732; dx=-1.2731444466940365E-8 evalInputDelta=4.351235543401202
Fitness changed from -74.73275430364612 to -79.08398984704732
Iteration 11 complete. Error: -79.08398984704732 Total: 3.0891; Orientation: 0.1432; Line Search: 2.2521
Adding measurement 7d7eabb2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.994e+03, gradient 1.626e-04, dot -0.124; [f179b27f-ca36-438a-b310-7ef413dfb443 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -79.08398984704732, -17.87728018004986, -16.284182180028186, -15.245563938357398
LBFGS Accumulation History: 3 points
Removed measurement 7d7eabb2 to history. Total: 3
th(0)=-79.08398984704732;dx=-2.5847956630329464E-8
New Minimum: -79.08398984704732 > -80.3468529887347
END: th(1.5120000000000006E8)=-80.3468529887347; dx=-2.9931379661774864E-9 evalInputDelta=1.262863141687376
Fitness changed from -79.08398984704732 to -80.3468529887347
Iteration 12 complete. Error: -80.3468529887347 Total: 1.8769; Orientation: 0.1387; Line Search: 1.1967
Adding measurement 35074828 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.048e+04, gradient 1.524e-04, dot -0.125; [f179b27f-ca36-438a-b310-7ef413dfb443 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -80.3468529887347, -17.87728018004986, -16.284182180028186, -15.245563938357398
LBFGS Accumulation History: 3 points
Removed measurement 35074828 to history. Total: 3
th(0)=-80.3468529887347;dx=-2.2732020775186264E-8
Armijo: th(3.2575052513282096E8)=-79.2840667873245; dx=1.363820166391161E-9 evalInputDelta=-1.0627862014101908
New Minimum: -80.3468529887347 > -81.32935438754893
END: th(1.6287526256641048E8)=-81.32935438754893; dx=-1.4467964647334858E-9 evalInputDelta=0.9825013988142359
Fitness changed from -80.3468529887347 to -81.32935438754893
Iteration 13 complete. Error: -81.32935438754893 Total: 2.4277; Orientation: 0.1373; Line Search: 1.7355
Adding measurement 372787b5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.463e+03, gradient 1.351e-04, dot -0.120; [f179b27f-ca36-438a-b310-7ef413dfb443 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -81.32935438754893, -17.87728018004986, -16.284182180028186, -15.245563938357398
LBFGS Accumulation History: 3 points
Removed measurement 372787b5 to history. Total: 3
th(0)=-81.32935438754893;dx=-1.7842922822650185E-8
Armijo: th(3.509041158211263E8)=-80.22556342186495; dx=1.344017299700262E-9 evalInputDelta=-1.1037909656839844
New Minimum: -81.32935438754893 > -82.00519554925192
END: th(1.7545205791056314E8)=-82.00519554925192; dx=-9.084188638340821E-10 evalInputDelta=0.6758411617029907
Fitness changed from -81.32935438754893 to -82.00519554925192
Iteration 14 complete. Error: -82.00519554925192 Total: 2.4874; Orientation: 0.1429; Line Search: 1.8015
Adding measurement 2ad35c9f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.673e+03, gradient 1.177e-04, dot -0.120; [f179b27f-ca36-438a-b310-7ef413dfb443 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -82.00519554925192, -17.87728018004986, -16.284182180028186, -15.245563938357398
LBFGS Accumulation History: 3 points
Removed measurement 2ad35c9f to history. Total: 3
th(0)=-82.00519554925192;dx=-1.3618819915069264E-8
Armijo: th(3.7800000000000024E8)=-81.03487462946009; dx=8.214363870011029E-10 evalInputDelta=-0.9703209197918312
New Minimum: -82.00519554925192 > -82.65557999353209
END: th(1.8900000000000012E8)=-82.65557999353209; dx=-7.936660194783629E-10 evalInputDelta=0.6503844442801636
Fitness changed from -82.00519554925192 to -82.65557999353209
Iteration 15 complete. Error: -82.65557999353209 Total: 2.4670; Orientation: 0.1480; Line Search: 1.7371
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -82.65557999353209 (> -Infinity) after 38.583s (< 5400.000s)
Returns
-82.65557999353209