BasicOptimizer.scala:89 executed in 81.09 seconds (1.488 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 2714997776961
Reset training subject: 2715797636057
Adding measurement 59f1e9b5 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-44.726546414929835;dx=-0.017970586729360635
New Minimum: -44.726546414929835 > -44.727985952680676
WOLFE (weak): th(2.154434690031884)=-44.727985952680676; dx=-0.017992092687501453 evalInputDelta=0.0014395377508407137
New Minimum: -44.727985952680676 > -44.72944533624741
WOLFE (weak): th(4.308869380063768)=-44.72944533624741; dx=-0.01798529098086838 evalInputDelta=0.0028989213175734108
New Minimum: -44.72944533624741 > -44.735260696734095
WOLFE (weak): th(12.926608140191302)=-44.735260696734095; dx=-0.017931697272652484 evalInputDelta=0.008714281804259372
New Minimum: -44.735260696734095 > -44.76091267563799
WOLFE (weak): th(51.70643256076521)=-44.76091267563799; dx=-0.01786465083171257 evalInputDelta=0.03436626070815407
New Minimum: -44.76091267563799 > -44.8900194288022
WOLFE (weak): th(258.53216280382605)=-44.8900194288022; dx=-0.017367525247444147 evalInputDelta=0.16347301387236257
New Minimum: -44.8900194288022 > -45.600631612994356
END: th(1551.1929768229563)=-45.600631612994356; dx=-0.014907379042254412 evalInputDelta=0.8740851980645203
Fitness changed from -44.726546414929835 to -45.600631612994356
Iteration 1 complete. Error: -45.600631612994356 Total: 5.8893; Orientation: 0.1999; Line Search: 3.9076
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 63c96f95 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-45.600631612994356;dx=-0.016907063399100156
New Minimum: -45.600631612994356 > -47.727578802646626
END: th(3341.943960201201)=-47.727578802646626; dx=-0.012131217655920557 evalInputDelta=2.1269471896522703
Fitness changed from -45.600631612994356 to -47.727578802646626
Iteration 2 complete. Error: -47.727578802646626 Total: 1.4357; Orientation: 0.0175; Line Search: 0.9531
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 22faeded to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-47.727578802646626;dx=-0.017442698494620053
New Minimum: -47.727578802646626 > -54.91060095182606
END: th(7200.000000000001)=-54.91060095182606; dx=-0.008529370003214323 evalInputDelta=7.183022149179436
Fitness changed from -47.727578802646626 to -54.91060095182606
Iteration 3 complete. Error: -54.91060095182606 Total: 1.3823; Orientation: 0.0185; Line Search: 0.9422
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement b4dd877 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.145e+04, gradient 1.347e-01, dot -0.448; [92b35936-70b3-470a-8c77-14d087d04edf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -54.91060095182606, -47.727578802646626, -45.600631612994356, -44.726546414929835
LBFGS Accumulation History: 3 points
Removed measurement b4dd877 to history. Total: 3
th(0)=-54.91060095182606;dx=-0.01808603588969565
New Minimum: -54.91060095182606 > -65.67413404924858
END: th(15511.929768229566)=-65.67413404924858; dx=-0.004149764790930585 evalInputDelta=10.763533097422517
Fitness changed from -54.91060095182606 to -65.67413404924858
Iteration 4 complete. Error: -65.67413404924858 Total: 2.0137; Orientation: 0.5916; Line Search: 0.9530
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 7c3d4b2e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.748e+03, gradient 1.325e-01, dot -0.225; [92b35936-70b3-470a-8c77-14d087d04edf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -65.67413404924858, -47.727578802646626, -45.600631612994356, -44.726546414929835
LBFGS Accumulation History: 3 points
Removed measurement 7c3d4b2e to history. Total: 3
th(0)=-65.67413404924858;dx=-0.017524370711900092
New Minimum: -65.67413404924858 > -76.60923797885303
END: th(33419.43960201202)=-76.60923797885303; dx=-0.0017430310538927884 evalInputDelta=10.935103929604452
Fitness changed from -65.67413404924858 to -76.60923797885303
Iteration 5 complete. Error: -76.60923797885303 Total: 2.2058; Orientation: 0.6051; Line Search: 1.1369
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 611bbfd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.396e+03, gradient 1.199e-01, dot -0.140; [92b35936-70b3-470a-8c77-14d087d04edf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -76.60923797885303, -47.727578802646626, -45.600631612994356, -44.726546414929835
LBFGS Accumulation History: 3 points
Removed measurement 611bbfd to history. Total: 3
th(0)=-76.60923797885303;dx=-0.014351275871965632
Armijo: th(72000.00000000003)=-71.51794551540205; dx=-6.172733247250636E-4 evalInputDelta=-5.0912924634509835
Armijo: th(36000.000000000015)=-75.91857365655378; dx=-0.0019345342667540185 evalInputDelta=-0.6906643222992557
New Minimum: -76.60923797885303 > -79.2236159142579
END: th(12000.000000000005)=-79.2236159142579; dx=-0.006228899478823404 evalInputDelta=2.6143779354048746
Fitness changed from -76.60923797885303 to -79.2236159142579
Iteration 6 complete. Error: -79.2236159142579 Total: 3.0883; Orientation: 0.4315; Line Search: 2.0088
Adding measurement 3008a32b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.206e+03, gradient 1.133e-01, dot -0.177; [92b35936-70b3-470a-8c77-14d087d04edf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -79.2236159142579, -47.727578802646626, -45.600631612994356, -44.726546414929835
LBFGS Accumulation History: 3 points
Removed measurement 3008a32b to history. Total: 3
th(0)=-79.2236159142579;dx=-0.012809555043030562
New Minimum: -79.2236159142579 > -83.14037616861698
END: th(25853.216280382618)=-83.14037616861698; dx=-0.00301348210709184 evalInputDelta=3.916760254359076
Fitness changed from -79.2236159142579 to -83.14037616861698
Iteration 7 complete. Error: -83.14037616861698 Total: 1.8588; Orientation: 0.4282; Line Search: 1.0051
Adding measurement 2cf5b288 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.283e+03, gradient 1.048e-01, dot -0.170; [92b35936-70b3-470a-8c77-14d087d04edf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -83.14037616861698, -47.727578802646626, -45.600631612994356, -44.726546414929835
LBFGS Accumulation History: 3 points
Removed measurement 2cf5b288 to history. Total: 3
th(0)=-83.14037616861698;dx=-0.010960541521627246
New Minimum: -83.14037616861698 > -84.9533745216325
END: th(55699.06600335338)=-84.9533745216325; dx=-0.0011434388632039804 evalInputDelta=1.8129983530155158
Fitness changed from -83.14037616861698 to -84.9533745216325
Iteration 8 complete. Error: -84.9533745216325 Total: 1.9172; Orientation: 0.4277; Line Search: 1.0475
Adding measurement 3e103a6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.532e+03, gradient 9.412e-02, dot -0.124; [92b35936-70b3-470a-8c77-14d087d04edf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -84.9533745216325, -47.727578802646626, -45.600631612994356, -44.726546414929835
LBFGS Accumulation History: 3 points
Removed measurement 3e103a6 to history. Total: 3
th(0)=-84.9533745216325;dx=-0.008833847748225073
New Minimum: -84.9533745216325 > -87.04807299671313
END: th(120000.00000000007)=-87.04807299671313; dx=-3.593814425601222E-4 evalInputDelta=2.0946984750806337
Fitness changed from -84.9533745216325 to -87.04807299671313
Iteration 9 complete. Error: -87.04807299671313 Total: 2.2031; Orientation: 0.4845; Line Search: 1.2535
Adding measurement 28242ee to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.935e+03, gradient 8.270e-02, dot -0.074; [92b35936-70b3-470a-8c77-14d087d04edf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -87.04807299671313, -47.727578802646626, -45.600631612994356, -44.726546414929835
LBFGS Accumulation History: 3 points
Removed measurement 28242ee to h
...skipping 12024 bytes...
09c1a to history. Total: 3
th(0)=-133.74864432862134;dx=-0.0017176495362566683
Armijo: th(453280.15953249845)=-129.47587691504685; dx=-1.5366744253818293E-4 evalInputDelta=-4.272767413574485
New Minimum: -133.74864432862134 > -142.0506206365744
END: th(226640.07976624923)=-142.0506206365744; dx=-4.214339055374747E-4 evalInputDelta=8.301976307953055
Fitness changed from -133.74864432862134 to -142.0506206365744
Iteration 23 complete. Error: -142.0506206365744 Total: 2.3248; Orientation: 0.4045; Line Search: 1.4796
Adding measurement 94679b2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.506e+05, gradient 4.261e-02, dot -0.160; [92b35936-70b3-470a-8c77-14d087d04edf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -142.0506206365744, -47.727578802646626, -45.600631612994356, -44.726546414929835
LBFGS Accumulation History: 3 points
Removed measurement 94679b2 to history. Total: 3
th(0)=-142.0506206365744;dx=-0.001804080305530051
Armijo: th(488281.2500000006)=-125.03921626050166; dx=-1.3243652090916025E-4 evalInputDelta=-17.011404376072733
New Minimum: -142.0506206365744 > -142.46450242541502
END: th(244140.6250000003)=-142.46450242541502; dx=-3.841660569485877E-4 evalInputDelta=0.41388178884062654
Fitness changed from -142.0506206365744 to -142.46450242541502
Iteration 24 complete. Error: -142.46450242541502 Total: 2.3897; Orientation: 0.4846; Line Search: 1.4625
Adding measurement 95415d2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.355e+05, gradient 4.003e-02, dot -0.153; [92b35936-70b3-470a-8c77-14d087d04edf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -142.46450242541502, -47.727578802646626, -45.600631612994356, -44.726546414929835
LBFGS Accumulation History: 3 points
Removed measurement 95415d2 to history. Total: 3
th(0)=-142.46450242541502;dx=-0.0015943994143187714
Armijo: th(525985.0317460661)=-126.03357685777641; dx=-1.2077164171179561E-4 evalInputDelta=-16.43092556763861
Armijo: th(262992.51587303303)=-142.28656441349412; dx=-3.490892828562592E-4 evalInputDelta=-0.17793801192090086
New Minimum: -142.46450242541502 > -144.85902295081803
END: th(87664.17195767768)=-144.85902295081803; dx=-8.134901838647848E-4 evalInputDelta=2.39452052540301
Fitness changed from -142.46450242541502 to -144.85902295081803
Iteration 25 complete. Error: -144.85902295081803 Total: 2.9231; Orientation: 0.5573; Line Search: 1.9438
<a id="p-19"></a>Iteration 25
<a id="p-18"></a>
Adding measurement 6781a0dd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.258e+05, gradient 3.862e-02, dot -0.171; [92b35936-70b3-470a-8c77-14d087d04edf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -144.85902295081803, -47.727578802646626, -45.600631612994356, -44.726546414929835
LBFGS Accumulation History: 3 points
Removed measurement 6781a0dd to history. Total: 3
th(0)=-144.85902295081803;dx=-0.001482754147595377
Armijo: th(188866.73313854108)=-142.38007436195545; dx=-5.061487516428908E-4 evalInputDelta=-2.478948588862579
New Minimum: -144.85902295081803 > -145.98263765501096
END: th(94433.36656927054)=-145.98263765501096; dx=-7.632766158627359E-4 evalInputDelta=1.1236147041929314
Fitness changed from -144.85902295081803 to -145.98263765501096
Iteration 26 complete. Error: -145.98263765501096 Total: 2.3613; Orientation: 0.4053; Line Search: 1.4944
Adding measurement a19fd4b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.636e+05, gradient 3.856e-02, dot -0.188; [92b35936-70b3-470a-8c77-14d087d04edf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -145.98263765501096, -47.727578802646626, -45.600631612994356, -44.726546414929835
LBFGS Accumulation History: 3 points
Removed measurement a19fd4b to history. Total: 3
th(0)=-145.98263765501096;dx=-0.0014765639024951949
Armijo: th(203450.52083333363)=-143.27609278714772; dx=-4.523244180135287E-4 evalInputDelta=-2.7065448678632436
New Minimum: -145.98263765501096 > -146.82357315665502
END: th(101725.26041666682)=-146.82357315665502; dx=-7.309554763668348E-4 evalInputDelta=0.8409355016440543
Fitness changed from -145.98263765501096 to -146.82357315665502
Iteration 27 complete. Error: -146.82357315665502 Total: 3.0672; Orientation: 0.4089; Line Search: 2.2042
Adding measurement 5d881e63 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.807e+06, gradient 3.800e-02, dot -0.195; [92b35936-70b3-470a-8c77-14d087d04edf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -146.82357315665502, -47.727578802646626, -45.600631612994356, -44.726546414929835
LBFGS Accumulation History: 3 points
Removed measurement 5d881e63 to history. Total: 3
th(0)=-146.82357315665502;dx=-0.0014349861586005989
Armijo: th(219160.42989419424)=-140.84577524027128; dx=-4.23457115399556E-4 evalInputDelta=-5.977797916383736
Armijo: th(109580.21494709712)=-146.12341971757422; dx=-6.891134206800623E-4 evalInputDelta=-0.7001534390807933
New Minimum: -146.82357315665502 > -147.34205827791567
END: th(36526.73831569904)=-147.34205827791567; dx=-9.712995834624602E-4 evalInputDelta=0.5184851212606532
Fitness changed from -146.82357315665502 to -147.34205827791567
Iteration 28 complete. Error: -147.34205827791567 Total: 2.7578; Orientation: 0.4179; Line Search: 1.9226
Adding measurement 41ce0841 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.682e+06, gradient 3.768e-02, dot -0.199; [92b35936-70b3-470a-8c77-14d087d04edf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -147.34205827791567, -47.727578802646626, -45.600631612994356, -44.726546414929835
LBFGS Accumulation History: 3 points
Removed measurement 41ce0841 to history. Total: 3
th(0)=-147.34205827791567;dx=-0.001408457969554968
Armijo: th(78694.4721410588)=-145.8007692031921; dx=-8.027060238916371E-4 evalInputDelta=-1.5412890747235792
Armijo: th(39347.2360705294)=-147.06329541005428; dx=-9.497811028548215E-4 evalInputDelta=-0.2787628678613885
Armijo: th(13115.745356843132)=-147.3056279296177; dx=-0.0010686303661448177 evalInputDelta=-0.036430348297955106
Armijo: th(3278.936339210783)=-147.33504435719826; dx=-0.0011261643262345516 evalInputDelta=-0.0070139207174122475
Armijo: th(655.7872678421566)=-147.34063092599447; dx=-0.0011474951955977404 evalInputDelta=-0.001427351921194031
Armijo: th(109.29787797369276)=-147.34179476783248; dx=-0.0011521388608841511 evalInputDelta=-2.635100831867021E-4
Armijo: th(15.613982567670394)=-147.3420143596012; dx=-0.0011543367392428477 evalInputDelta=-4.3918314474922227E-5
Armijo: th(1.9517478209587993)=-147.3420538860848; dx=-0.0011524014905366538 evalInputDelta=-4.3918308563206665E-6
Armijo: th(0.21686086899542215)=-147.34205827791567; dx=-0.0011542578368977392 evalInputDelta=0.0
Armijo: th(0.021686086899542215)=-147.34205827791567; dx=-0.0011542584863330782 evalInputDelta=0.0
Armijo: th(0.0019714624454129285)=-147.34205827791567; dx=-0.0011526644591289504 evalInputDelta=0.0
Armijo: th(1.6428853711774404E-4)=-147.34205827791567; dx=-0.001153674324580808 evalInputDelta=0.0
Armijo: th(1.2637579778288002E-5)=-147.34205827791567; dx=-0.001154685033253017 evalInputDelta=0.0
END: th(9.026842698777144E-7)=-147.34205827791567; dx=-0.0011544025231630915 evalInputDelta=0.0
Fitness changed from -147.34205827791567 to -147.34205827791567
Static Iteration Total: 8.0823; Orientation: 0.4078; Line Search: 7.2578
Iteration 29 failed. Error: -147.34205827791567
Previous Error: 0.0 -> -147.34205827791567
Retrying iteration 29
Reset training subject: 2794276089677
Adding measurement 39970340 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-147.34205827791567;dx=-0.0014114530460578902
END: th(1.944774305170651E-6)=-147.34205827791567; dx=-0.0011560731993628566 evalInputDelta=0.0
Fitness changed from -147.34205827791567 to -147.34205827791567
Static Iteration Total: 1.7911; Orientation: 0.0167; Line Search: 0.9663
Iteration 30 failed. Error: -147.34205827791567
Previous Error: 0.0 -> -147.34205827791567
Optimization terminated 30
Final threshold in iteration 30: -147.34205827791567 (> -Infinity) after 81.070s (< 3600.000s)
Returns
-147.34205827791567