BasicOptimizer.scala:89 executed in 64.11 seconds (1.570 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 144893683871400
Reset training subject: 144895448029900
Adding measurement 7fca42e1 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-3.958760033344603;dx=-8.640706689079507E-6
New Minimum: -3.958760033344603 > -3.9587816234946973
WOLFE (weak): th(2.154434690031884)=-3.9587816234946973; dx=-8.453451074243223E-6 evalInputDelta=2.159015009439713E-5
New Minimum: -3.9587816234946973 > -3.958802850670332
WOLFE (weak): th(4.308869380063768)=-3.958802850670332; dx=-8.505974986319037E-6 evalInputDelta=4.281732572897212E-5
New Minimum: -3.958802850670332 > -3.9588895724790127
WOLFE (weak): th(12.926608140191302)=-3.9588895724790127; dx=-8.512006093663554E-6 evalInputDelta=1.2953913440982845E-4
New Minimum: -3.9588895724790127 > -3.9592790714855957
WOLFE (weak): th(51.70643256076521)=-3.9592790714855957; dx=-8.510290493709852E-6 evalInputDelta=5.190381409927447E-4
New Minimum: -3.9592790714855957 > -3.961355994469447
WOLFE (weak): th(258.53216280382605)=-3.961355994469447; dx=-8.507002219732915E-6 evalInputDelta=0.0025959611248440595
New Minimum: -3.961355994469447 > -3.974347232319545
WOLFE (weak): th(1551.1929768229563)=-3.974347232319545; dx=-8.490961805734965E-6 evalInputDelta=0.015587198974941874
New Minimum: -3.974347232319545 > -4.068255718215153
WOLFE (weak): th(10858.350837760694)=-4.068255718215153; dx=-8.536059203437353E-6 evalInputDelta=0.10949568487054995
New Minimum: -4.068255718215153 > -4.8519210478567025
WOLFE (weak): th(86866.80670208555)=-4.8519210478567025; dx=-8.462328437473647E-6 evalInputDelta=0.8931610145120996
New Minimum: -4.8519210478567025 > -9.974724976972517
END: th(781801.26031877)=-9.974724976972517; dx=-3.5257883128640627E-6 evalInputDelta=6.015964943627914
Fitness changed from -3.958760033344603 to -9.974724976972517
Iteration 1 complete. Error: -9.974724976972517 Total: 12.3067; Orientation: 0.0156; Line Search: 8.8349
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 180cd76d to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-9.974724976972517;dx=-1.1304772457063243E-5
New Minimum: -9.974724976972517 > -20.72868682058955
END: th(1684339.7559414052)=-20.72868682058955; dx=-1.5647184813346083E-6 evalInputDelta=10.753961843617034
Fitness changed from -9.974724976972517 to -20.72868682058955
Iteration 2 complete. Error: -20.72868682058955 Total: 2.6142; Orientation: 0.0146; Line Search: 1.6818
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 210f6573 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-20.72868682058955;dx=-1.5175487847143184E-5
New Minimum: -20.72868682058955 > -27.4711909674986
WOLF (strong): th(3628800.0)=-27.4711909674986; dx=1.143258541468182E-6 evalInputDelta=6.742504146909049
New Minimum: -27.4711909674986 > -30.91892631246867
END: th(1814400.0)=-30.91892631246867; dx=-5.173901596757666E-7 evalInputDelta=10.190239491879119
Fitness changed from -20.72868682058955 to -30.91892631246867
Iteration 3 complete. Error: -30.91892631246867 Total: 3.5381; Orientation: 0.0144; Line Search: 2.6087
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 3a94ab2f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.967e+04, gradient 4.234e-03, dot -0.249; [84d882cc-756f-4163-8c8f-9f03dabdc8e7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -30.91892631246867, -20.72868682058955, -9.974724976972517, -3.958760033344603
LBFGS Accumulation History: 3 points
Removed measurement 3a94ab2f to history. Total: 3
th(0)=-30.91892631246867;dx=-1.7869985867655715E-5
New Minimum: -30.91892631246867 > -31.98784844208713
WOLF (strong): th(3909006.30159385)=-31.98784844208713; dx=2.069450526735384E-6 evalInputDelta=1.0689221296184606
New Minimum: -31.98784844208713 > -38.8239011883885
WOLF (strong): th(1954503.150796925)=-38.8239011883885; dx=7.794461675395009E-7 evalInputDelta=7.904974875919827
New Minimum: -38.8239011883885 > -39.25307393663229
END: th(651501.0502656417)=-39.25307393663229; dx=-5.672230672221835E-6 evalInputDelta=8.33414762416362
Fitness changed from -30.91892631246867 to -39.25307393663229
Iteration 4 complete. Error: -39.25307393663229 Total: 4.8093; Orientation: 0.2759; Line Search: 3.6030
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 18dfc803 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.538e+04, gradient 3.619e-03, dot -0.284; [84d882cc-756f-4163-8c8f-9f03dabdc8e7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -39.25307393663229, -20.72868682058955, -9.974724976972517, -3.958760033344603
LBFGS Accumulation History: 3 points
Removed measurement 18dfc803 to history. Total: 3
th(0)=-39.25307393663229;dx=-1.295092462217999E-5
New Minimum: -39.25307393663229 > -45.6415780798608
END: th(1403616.4632845044)=-45.6415780798608; dx=-2.1389755381772206E-6 evalInputDelta=6.388504143228509
Fitness changed from -39.25307393663229 to -45.6415780798608
Iteration 5 complete. Error: -45.6415780798608 Total: 2.9070; Orientation: 0.3065; Line Search: 1.7187
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 760ce152 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.819e+04, gradient 4.146e-03, dot -0.277; [84d882cc-756f-4163-8c8f-9f03dabdc8e7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -45.6415780798608, -20.72868682058955, -9.974724976972517, -3.958760033344603
LBFGS Accumulation History: 3 points
Removed measurement 760ce152 to history. Total: 3
th(0)=-45.6415780798608;dx=-1.705274525908827E-5
Armijo: th(3024000.0)=-45.105729802271284; dx=1.2814594657600623E-6 evalInputDelta=-0.5358482775895155
New Minimum: -45.6415780798608 > -50.2816963632911
WOLF (strong): th(1512000.0)=-50.2816963632911; dx=6.365111460924108E-7 evalInputDelta=4.640118283430297
New Minimum: -50.2816963632911 > -51.8706250875558
END: th(504000.0)=-51.8706250875558; dx=-5.8173233730408864E-6 evalInputDelta=6.2290470076950015
Fitness changed from -45.6415780798608 to -51.8706250875558
Iteration 6 complete. Error: -51.8706250875558 Total: 4.7182; Orientation: 0.3148; Line Search: 3.4717
Adding measurement 348120c4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.005e+04, gradient 3.311e-03, dot -0.343; [84d882cc-756f-4163-8c8f-9f03dabdc8e7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -51.8706250875558, -20.72868682058955, -9.974724976972517, -3.958760033344603
LBFGS Accumulation History: 3 points
Removed measurement 348120c4 to history. Total: 3
th(0)=-51.8706250875558;dx=-1.07926142839349E-5
New Minimum: -51.8706250875558 > -57.33005058158735
END: th(1085835.0837760693)=-57.33005058158735; dx=-3.637376194646658E-6 evalInputDelta=5.459425494031549
Fitness changed from -51.8706250875558 to -57.33005058158735
Iteration 7 complete. Error: -57.33005058158735 Total: 3.1059; Orientation: 0.2947; Line Search: 1.9966
Adding measurement 68efc6de to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.100e+04, gradient 3.699e-03, dot -0.343; [84d882cc-756f-4163-8c8f-9f03dabdc8e7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -57.33005058158735, -20.72868682058955, -9.974724976972517, -3.958760033344603
LBFGS Accumulation History: 3 points
Removed measurement 68efc6de to history. Total: 3
th(0)=-57.33005058158735;dx=-1.3652649779578116E-5
Armijo: th(2339360.7721408405)=-57.045161935545714; dx=4.7957432845976E-7 evalInputDelta=-0.28488864604163666
New Minimum: -57.33005058158735 > -60.59905277215965
END: th(1169680.3860704203)=-60.59905277215965; dx=-5.583033227053073E-7 evalInputDelta=3.2690021905723015
Fitness changed from -57.33005058158735 to -60.59905277215965
Iteration 8 complete. Error: -60.59905277215965 Total: 3.7111; Orientation: 0.2781; Line Search: 2.6130
Adding measurement f6b6f19 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.176e+04, gradient 4.286e-03, dot -0.278; [84d882cc-756f-4163-8c8f-9f03dabdc8e7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -60.59905277215965, -20.72868682058955, -9.974724976972517, -3.958760033344603
LBFGS Accumulation History: 3 points
Removed measurement f6b6f19 to history. Total: 3
th(0)=-60.59905277215965;dx=-1.8288696261010847E-5
Armijo: th(2520000.0)=-54.1577959057974; dx=2.8733099704788883E-6 evalInputDelta=-6.441256866362252
New Minimum: -60.59905277215965 > -63.387445398878114
WOLF (strong): th(1260000.0)=-63.387445398878114; dx=2.895564229311381E-6 evalInputDelta=2.788392626718462
New Minimum: -63.387445398878114 > -66.15060944110623
END: th(420000.0)=-66.15060944110623; dx=-6.4771435567255034E-6 evalInputDelta=5.551556668946574
Fitness changed from -60.59905277215965 to -66.15060944110623
Iteration 9 complete. Error: -66.15060944110623 Total: 4.6409; Orientation: 0.2759; Line Search: 3.4947
Adding measurement 78d4ca21 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.660e+05, gradient 3.239e-03, dot -0.407; [84d882cc-756f-4163-8c8f-9f03dabdc8e7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -66.15060944110623, -20.72868682058955, -9.974724976972517, -3.958760033344603
LBFGS Accumulation History: 3 points
Removed measurement 78d4ca21 to history. Total: 3
th(0)=-66.15060944110623;dx=-1.0356480588980772E-5
New Minimum: -66.15060944110623 > -70.39110760228066
END: th(904862.5698133912)=-70.39110760228066; dx=-4.314271797347314E-6 evalInputDelta=4.240498161174429
Fitness changed from -66.15060944110623 to -70.39110760228066
Iteration 10 complete. Error: -70.39110760228066 Total: 2.9340; Orientation: 0.2664; Line Search: 1.7308
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 565e9708 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.802e+05, gradient 3.354e-03, dot -0.409; [84d882cc-756f-4163-8c8f-9f03dabdc8e7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -70.39110760228066, -20.72868682058955, -9.974724976972517, -3.958760033344603
LBFGS Accumulation History: 3 points
Removed measurement 565e9708 to history. Total: 3
th(0)=-70.39110760228066;dx=-1.1181281667459317E-5
New Minimum: -70.39110760228066 > -73.04674996233268
END: th(1949467.3101173672)=-73.04674996233268; dx=-9.423034233589007E-7 evalInputDelta=2.6556423600520276
Fitness changed from -70.39110760228066 to -73.04674996233268
Iteration 11 complete. Error: -73.04674996233268 Total: 3.0265; Orientation: 0.3003; Line Search: 1.7908
Adding measurement 24fa6e45 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.992e+05, gradient 4.580e-03, dot -0.281; [84d882cc-756f-4163-8c8f-9f03dabdc8e7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -73.04674996233268, -20.72868682058955, -9.974724976972517, -3.958760033344603
LBFGS Accumulation History: 3 points
Removed measurement 24fa6e45 to history. Total: 3
th(0)=-73.04674996233268;dx=-2.0922842146566408E-5
Armijo: th(4200000.0)=-53.40157203996587; dx=3.1404170819057044E-6 evalInputDelta=-19.645177922366813
Armijo: th(2100000.0)=-68.82413223192029; dx=4.499032410098956E-6 evalInputDelta=-4.222617730412395
New Minimum: -73.04674996233268 > -80.45680487014297
END: th(700000.0)=-80.45680487014297; dx=-1.8153467790367851E-6 evalInputDelta=7.4100549078102915
Fitness changed from -73.04674996233268 to -80.45680487014297
Iteration 12 complete. Error: -80.45680487014297 Total: 4.4952; Orientation: 0.2779; Line Search: 3.3968
Adding measurement 22264fa5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.564e+06, gradient 3.569e-03, dot -0.391; [84d882cc-756f-4163-8c8f-9f03dabdc8e7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -80.45680487014297, -20.72868682058955, -9.974724976972517, -3.958760033344603
LBFGS Accumulation History: 3 points
Removed measurement 22264fa5 to history. Total: 3
th(0)=-80.45680487014297;dx=-1.2673264703208292E-5
New Minimum: -80.45680487014297 > -83.0508520001657
WOLF (strong): th(1508104.2830223187)=-83.0508520001657; dx=1.1776502841776419E-7 evalInputDelta=2.59404713002273
New Minimum: -83.0508520001657 > -84.37699843678605
END: th(754052.1415111594)=-84.37699843678605; dx=-3.1522648194908173E-6 evalInputDelta=3.9201935666430785
Fitness changed from -80.45680487014297 to -84.37699843678605
Iteration 13 complete. Error: -84.37699843678605 Total: 3.7579; Orientation: 0.2644; Line Search: 2.5352
Adding measurement 45c732e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.690e+08, gradient 3.446e-03, dot -0.405; [84d882cc-756f-4163-8c8f-9f03dabdc8e7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -84.37699843678605, -20.72868682058955, -9.974724976972517, -3.958760033344603
LBFGS Accumulation History: 3 points
Removed measurement 45c732e to history. Total: 3
th(0)=-84.37699843678605;dx=-1.184388227662626E-5
New Minimum: -84.37699843678605 > -86.09477444505947
END: th(1624556.091764473)=-86.09477444505947; dx=-3.246259161970523E-7 evalInputDelta=1.7177760082734181
Fitness changed from -84.37699843678605 to -86.09477444505947
Iteration 14 complete. Error: -86.09477444505947 Total: 2.7974; Orientation: 0.2742; Line Search: 1.6962
Adding measurement 9c0883c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.397e+07, gradient 4.544e-03, dot -0.299; [84d882cc-756f-4163-8c8f-9f03dabdc8e7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -86.09477444505947, -20.72868682058955, -9.974724976972517, -3.958760033344603
LBFGS Accumulation History: 3 points
Removed measurement 9c0883c to history. Total: 3
th(0)=-86.09477444505947;dx=-2.0611861788278532E-5
Armijo: th(3500000.000000001)=-65.23271408836685; dx=4.147365684615575E-6 evalInputDelta=-20.862060356692623
Armijo: th(1750000.0000000005)=-81.96704155609314; dx=5.152253364975887E-6 evalInputDelta=-4.127732888966335
New Minimum: -86.09477444505947 > -92.30189858434048
END: th(583333.3333333335)=-92.30189858434048; dx=-1.9582744771199136E-6 evalInputDelta=6.207124139281007
Fitness changed from -86.09477444505947 to -92.30189858434048
Iteration 15 complete. Error: -92.30189858434048 Total: 4.6490; Orientation: 0.2765; Line Search: 3.5511
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -92.30189858434048 (> -Infinity) after 64.110s (< 5400.000s)
Returns
-92.30189858434048