BasicOptimizer.scala:89 executed in 337.95 seconds (3.870 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 1390739429973
Reset training subject: 1391631387255
Adding measurement 21925238 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.05075269458128884;dx=-2.3370474920471693E-6
New Minimum: -0.05075269458128884 > -0.05075753913262155
WOLFE (weak): th(2.154434690031884)=-0.05075753913262155; dx=-2.3391474611674943E-6 evalInputDelta=4.844551332712499E-6
New Minimum: -0.05075753913262155 > -0.050762610276939825
WOLFE (weak): th(4.308869380063768)=-0.050762610276939825; dx=-2.338901105693265E-6 evalInputDelta=9.915695650987433E-6
New Minimum: -0.050762610276939825 > -0.050782735191199184
WOLFE (weak): th(12.926608140191302)=-0.050782735191199184; dx=-2.336207227970207E-6 evalInputDelta=3.004060991034646E-5
New Minimum: -0.050782735191199184 > -0.050873075654236675
WOLFE (weak): th(51.70643256076521)=-0.050873075654236675; dx=-2.336343427750588E-6 evalInputDelta=1.2038107294783801E-4
New Minimum: -0.050873075654236675 > -0.051354614444516555
WOLFE (weak): th(258.53216280382605)=-0.051354614444516555; dx=-2.3467937212675422E-6 evalInputDelta=6.019198632277178E-4
New Minimum: -0.051354614444516555 > -0.05443174876299565
WOLFE (weak): th(1551.1929768229563)=-0.05443174876299565; dx=-2.4030617774280588E-6 evalInputDelta=0.0036790541817068104
New Minimum: -0.05443174876299565 > -0.07897721827662262
WOLFE (weak): th(10858.350837760694)=-0.07897721827662262; dx=-2.604811607139611E-6 evalInputDelta=0.028224523695333778
New Minimum: -0.07897721827662262 > -0.23944473984372008
END: th(86866.80670208555)=-0.23944473984372008; dx=-1.576935169280872E-6 evalInputDelta=0.18869204526243125
Fitness changed from -0.05075269458128884 to -0.23944473984372008
Iteration 1 complete. Error: -0.23944473984372008 Total: 10.5668; Orientation: 0.0255; Line Search: 7.9597
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 2b4115ea to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-0.23944473984372008;dx=-7.516163256215403E-6
New Minimum: -0.23944473984372008 > -0.48435188356216363
END: th(187148.86177126726)=-0.48435188356216363; dx=-4.583871600418276E-8 evalInputDelta=0.24490714371844355
Fitness changed from -0.23944473984372008 to -0.48435188356216363
Iteration 2 complete. Error: -0.48435188356216363 Total: 2.6538; Orientation: 0.0216; Line Search: 1.7657
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 7f6518f6 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-0.48435188356216363;dx=-1.1323307408888834E-5
New Minimum: -0.48435188356216363 > -0.6361956873492827
WOLF (strong): th(403200.00000000006)=-0.6361956873492827; dx=6.04616372628976E-7 evalInputDelta=0.15184380378711904
New Minimum: -0.6361956873492827 > -0.9276112229999197
WOLF (strong): th(201600.00000000003)=-0.9276112229999197; dx=1.0962564558282525E-7 evalInputDelta=0.4432593394377561
END: th(67200.00000000001)=-0.8837954615858806; dx=-3.449811106260999E-6 evalInputDelta=0.399443578023717
Fitness changed from -0.48435188356216363 to -0.9276112229999197
Iteration 3 complete. Error: -0.9276112229999197 Total: 4.4181; Orientation: 0.0217; Line Search: 3.5230
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 270380a0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.922e+03, gradient 4.339e-03, dot -0.070; [5a987b76-acbf-43e6-9220-040500622bb4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.9276112229999197, -0.48435188356216363, -0.23944473984372008, -0.05075269458128884
LBFGS Accumulation History: 3 points
Removed measurement 270380a0 to history. Total: 3
th(0)=-0.9276112229999197;dx=-1.8670561245378354E-5
New Minimum: -0.9276112229999197 > -1.5742294153693914
END: th(144778.01117014262)=-1.5742294153693914; dx=-5.582807718191304E-7 evalInputDelta=0.6466181923694717
Fitness changed from -0.9276112229999197 to -1.5742294153693914
Iteration 4 complete. Error: -1.5742294153693914 Total: 3.0973; Orientation: 0.4724; Line Search: 1.7669
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 598b1732 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.837e+03, gradient 4.724e-03, dot -0.058; [5a987b76-acbf-43e6-9220-040500622bb4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.5742294153693914, -0.48435188356216363, -0.23944473984372008, -0.05075269458128884
LBFGS Accumulation History: 3 points
Removed measurement 598b1732 to history. Total: 3
th(0)=-1.5742294153693914;dx=-2.2135350636447842E-5
New Minimum: -1.5742294153693914 > -1.8267844626586036
WOLF (strong): th(311914.76961877884)=-1.8267844626586036; dx=1.4358405340952454E-6 evalInputDelta=0.2525550472892122
New Minimum: -1.8267844626586036 > -2.1622613462334592
WOLF (strong): th(155957.38480938942)=-2.1622613462334592; dx=1.916113914166017E-8 evalInputDelta=0.5880319308640678
END: th(51985.79493646314)=-2.115696559804615; dx=-5.6533499712727444E-6 evalInputDelta=0.5414671444352235
Fitness changed from -1.5742294153693914 to -2.1622613462334592
Iteration 5 complete. Error: -2.1622613462334592 Total: 4.9889; Orientation: 0.6073; Line Search: 3.5261
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 74551e5d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.703e+03, gradient 4.869e-03, dot -0.058; [5a987b76-acbf-43e6-9220-040500622bb4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.1622613462334592, -0.48435188356216363, -0.23944473984372008, -0.05075269458128884
LBFGS Accumulation History: 3 points
Removed measurement 74551e5d to history. Total: 3
th(0)=-2.1622613462334592;dx=-2.3593551798328885E-5
New Minimum: -2.1622613462334592 > -2.947276419311118
END: th(112000.00000000004)=-2.947276419311118; dx=-2.1568669467122084E-6 evalInputDelta=0.7850150730776586
Fitness changed from -2.1622613462334592 to -2.947276419311118
Iteration 6 complete. Error: -2.947276419311118 Total: 3.1393; Orientation: 0.4999; Line Search: 1.7815
Adding measurement 3b765c2b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.043e+03, gradient 4.795e-03, dot -0.065; [5a987b76-acbf-43e6-9220-040500622bb4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.947276419311118, -0.48435188356216363, -0.23944473984372008, -0.05075269458128884
LBFGS Accumulation History: 3 points
Removed measurement 3b765c2b to history. Total: 3
th(0)=-2.947276419311118;dx=-2.2934245155104788E-5
New Minimum: -2.947276419311118 > -3.20479500205834
WOLF (strong): th(241296.68528357107)=-3.20479500205834; dx=1.7750176202806423E-6 evalInputDelta=0.25751858274722217
New Minimum: -3.20479500205834 > -3.5665086125219534
END: th(120648.34264178554)=-3.5665086125219534; dx=-2.1187571039460853E-7 evalInputDelta=0.6192321932108356
Fitness changed from -2.947276419311118 to -3.5665086125219534
Iteration 7 complete. Error: -3.5665086125219534 Total: 4.7903; Orientation: 1.1791; Line Search: 2.7776
Adding measurement 337756d2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.389e+03, gradient 5.695e-03, dot -0.057; [5a987b76-acbf-43e6-9220-040500622bb4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.5665086125219534, -0.48435188356216363, -0.23944473984372008, -0.05075269458128884
LBFGS Accumulation History: 3 points
Removed measurement 337756d2 to history. Total: 3
th(0)=-3.5665086125219534;dx=-3.234335285140812E-5
New Minimum: -3.5665086125219534 > -3.5818975394135837
WOLF (strong): th(259928.97468231575)=-3.5818975394135837; dx=2.914249788908467E-6 evalInputDelta=0.015388926891630295
New Minimum: -3.5818975394135837 > -4.207113529430945
WOLF (strong): th(129964.48734115787)=-4.207113529430945; dx=1.5834907014238352E-6 evalInputDelta=0.6406049169089916
New Minimum: -4.207113529430945 > -4.267468695808501
END: th(43321.49578038596)=-4.267468695808501; dx=-9.87683305989269E-6 evalInp
...skipping 50923 bytes...
to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.964e+04, gradient 5.927e-03, dot -0.085; [5a987b76-acbf-43e6-9220-040500622bb4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -21.851323443595106, -0.48435188356216363, -0.23944473984372008, -0.05075269458128884
LBFGS Accumulation History: 3 points
Removed measurement 7a7d8573 to history. Total: 3
th(0)=-21.851323443595106;dx=-3.5106131095362016E-5
New Minimum: -21.851323443595106 > -22.14964371040323
END: th(11907.48361530256)=-22.14964371040323; dx=-2.973277781960358E-5 evalInputDelta=0.2983202668081226
Fitness changed from -21.851323443595106 to -22.14964371040323
Iteration 45 complete. Error: -22.14964371040323 Total: 3.0936; Orientation: 0.4712; Line Search: 1.7872
<a id="p-27"></a>Iteration 45
<a id="p-26"></a>
Adding measurement 7bbb5756 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.401e+04, gradient 5.787e-03, dot -0.090; [5a987b76-acbf-43e6-9220-040500622bb4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -22.14964371040323, -0.48435188356216363, -0.23944473984372008, -0.05075269458128884
LBFGS Accumulation History: 3 points
Removed measurement 7bbb5756 to history. Total: 3
th(0)=-22.14964371040323;dx=-3.3470003302792315E-5
New Minimum: -22.14964371040323 > -22.72265846556745
END: th(25653.89577179411)=-22.72265846556745; dx=-2.628118847131239E-5 evalInputDelta=0.5730147551642197
Fitness changed from -22.14964371040323 to -22.72265846556745
Iteration 46 complete. Error: -22.72265846556745 Total: 3.1856; Orientation: 0.5108; Line Search: 1.7953
Adding measurement 3f95a39c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.771e+04, gradient 5.433e-03, dot -0.103; [5a987b76-acbf-43e6-9220-040500622bb4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -22.72265846556745, -0.48435188356216363, -0.23944473984372008, -0.05075269458128884
LBFGS Accumulation History: 3 points
Removed measurement 3f95a39c to history. Total: 3
th(0)=-22.72265846556745;dx=-2.950187030301313E-5
New Minimum: -22.72265846556745 > -23.68822585054259
END: th(55269.6429852155)=-23.68822585054259; dx=-2.1137980574970923E-5 evalInputDelta=0.9655673849751416
Fitness changed from -22.72265846556745 to -23.68822585054259
Iteration 47 complete. Error: -23.68822585054259 Total: 3.1247; Orientation: 0.4725; Line Search: 1.8096
Adding measurement 5352d515 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.850e+04, gradient 4.996e-03, dot -0.126; [5a987b76-acbf-43e6-9220-040500622bb4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -23.68822585054259, -0.48435188356216363, -0.23944473984372008, -0.05075269458128884
LBFGS Accumulation History: 3 points
Removed measurement 5352d515 to history. Total: 3
th(0)=-23.68822585054259;dx=-2.4938736269431972E-5
New Minimum: -23.68822585054259 > -25.19955925699856
END: th(119074.83615302564)=-25.19955925699856; dx=-1.573314652333269E-5 evalInputDelta=1.511333406455968
Fitness changed from -23.68822585054259 to -25.19955925699856
Iteration 48 complete. Error: -25.19955925699856 Total: 3.1015; Orientation: 0.4767; Line Search: 1.7878
Adding measurement 13c8bf27 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.316e+06, gradient 4.640e-03, dot -0.153; [5a987b76-acbf-43e6-9220-040500622bb4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -25.19955925699856, -0.48435188356216363, -0.23944473984372008, -0.05075269458128884
LBFGS Accumulation History: 3 points
Removed measurement 13c8bf27 to history. Total: 3
th(0)=-25.19955925699856;dx=-2.150840177787052E-5
New Minimum: -25.19955925699856 > -27.17199056291615
END: th(256538.95771794114)=-27.17199056291615; dx=-9.515558889817421E-6 evalInputDelta=1.972431305917592
Fitness changed from -25.19955925699856 to -27.17199056291615
Iteration 49 complete. Error: -27.17199056291615 Total: 3.7458; Orientation: 0.4700; Line Search: 2.4306
Adding measurement 4fb0a436 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 7.710e+04, gradient 5.134e-03, dot 0.167; [5a987b76-acbf-43e6-9220-040500622bb4 = 1.000/1.000e+00]
th(0)=-27.17199056291615;dx=-66.11809362543679
Armijo: th(552696.429852155)=-5.647140465453609; dx=1.0199726479422797E-5 evalInputDelta=-21.52485009746254
Armijo: th(276348.2149260775)=-5.649006495806806; dx=2.0412908416610324E-5 evalInputDelta=-21.522984067109345
Armijo: th(92116.07164202584)=-5.646655564908859; dx=6.130184298679348E-5 evalInputDelta=-21.525334998007292
Armijo: th(23029.01791050646)=-5.641424424482166; dx=2.4553013852647793E-4 evalInputDelta=-21.530566138433983
Armijo: th(4605.803582101292)=-5.6535718187182615; dx=0.001221633207829366 evalInputDelta=-21.518418744197888
Armijo: th(767.6339303502153)=-5.654329604717589; dx=0.007330231923485017 evalInputDelta=-21.51766095819856
Armijo: th(109.66199005003077)=-5.6955347236043785; dx=0.052003202973572894 evalInputDelta=-21.47645583931177
Armijo: th(13.707748756253846)=-6.112220734110032; dx=0.41143318704660775 evalInputDelta=-21.059769828806118
Armijo: th(1.5230831951393162)=-11.685139954159817; dx=4.789830577901631 evalInputDelta=-15.486850608756333
New Minimum: -27.17199056291615 > -27.725555141531075
WOLF (strong): th(0.15230831951393162)=-27.725555141531075; dx=4.4973005512033035 evalInputDelta=0.5535645786149246
END: th(0.013846210864902874)=-27.685079382810898; dx=-28.993910741095277 evalInputDelta=0.5130888198947474
Fitness changed from -27.17199056291615 to -27.725555141531075
Iteration 50 complete. Error: -27.725555141531075 Total: 12.1297; Orientation: 0.4752; Line Search: 10.8188
<a id="p-29"></a>Iteration 50
<a id="p-28"></a>
Adding measurement 465bcc22 to history. Total: 4
Accepted: LBFGS Orientation magnitude: 2.788e+04, gradient 5.642e-03, dot 0.131; [5a987b76-acbf-43e6-9220-040500622bb4 = 1.000/1.000e+00]
th(0)=-27.725555141531075;dx=-20.627301646644902
Armijo: th(0.029830757012843125)=-27.577531080409432; dx=1.0536520038480348 evalInputDelta=-0.14802406112164235
Armijo: th(0.014915378506421563)=-27.652357145623025; dx=0.9214918717014192 evalInputDelta=-0.07319799590804976
Armijo: th(0.004971792835473854)=-27.701500301694914; dx=0.7906982842208399 evalInputDelta=-0.02405483983616108
Armijo: th(0.0012429482088684636)=-27.719601838055535; dx=0.7834410789782702 evalInputDelta=-0.005953303475539684
Armijo: th(2.485896417736927E-4)=-27.72436268777363; dx=0.7731925089902592 evalInputDelta=-0.001192453757443701
Armijo: th(4.1431606962282115E-5)=-27.7253561395687; dx=0.7653975539705052 evalInputDelta=-1.9900196237543355E-4
Armijo: th(5.9188009946117306E-6)=-27.725523554013368; dx=0.761125244823234 evalInputDelta=-3.158751770726553E-5
Armijo: th(7.398501243264663E-7)=-27.72554882403041; dx=0.7660672564872202 evalInputDelta=-6.317500663044484E-6
Armijo: th(8.220556936960737E-8)=-27.725553562156044; dx=0.7665408939927375 evalInputDelta=-1.5793750307580012E-6
Armijo: th(8.220556936960737E-9)=-27.725555141531075; dx=0.7591605980529368 evalInputDelta=0.0
Armijo: th(7.473233579055215E-10)=-27.725553562156044; dx=0.7621114537111013 evalInputDelta=-1.5793750307580012E-6
MIN ALPHA (6.22769464921268E-11): th(0.0)=-27.725555141531075
Fitness changed from -27.725555141531075 to -27.725555141531075
Static Iteration Total: 13.1813; Orientation: 0.5803; Line Search: 11.7396
Iteration 51 failed. Error: -27.725555141531075
Previous Error: 0.0 -> -27.725555141531075
Retrying iteration 51
Reset training subject: 1725023591275
Adding measurement 4b0496a9 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-27.725555141531075;dx=-3.180764842473542E-5
END: th(2.1544346900318838E-10)=-27.725555141531075; dx=-2.663413175188E-5 evalInputDelta=0.0
Fitness changed from -27.725555141531075 to -27.725555141531075
Static Iteration Total: 3.6674; Orientation: 0.0201; Line Search: 1.9542
Iteration 52 failed. Error: -27.725555141531075
Previous Error: 0.0 -> -27.725555141531075
Optimization terminated 52
Final threshold in iteration 52: -27.725555141531075 (> -Infinity) after 337.952s (< 3600.000s)
Returns
-27.725555141531075