BasicOptimizer.scala:89 executed in 8.75 seconds (0.000 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 155366511515500
Reset training subject: 155366641423900
Adding measurement 7ec083e2 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.7391182930342828;dx=-3.9678784812675736E-4
New Minimum: -0.7391182930342828 > -0.739818691396717
WOLFE (weak): th(2.154434690031884)=-0.739818691396717; dx=-3.99604963647765E-4 evalInputDelta=7.003983624341847E-4
New Minimum: -0.739818691396717 > -0.7405184002008788
WOLFE (weak): th(4.308869380063768)=-0.7405184002008788; dx=-3.893898974114309E-4 evalInputDelta=0.0014001071665959985
New Minimum: -0.7405184002008788 > -0.7433133762811734
WOLFE (weak): th(12.926608140191302)=-0.7433133762811734; dx=-3.9640297728561963E-4 evalInputDelta=0.004195083246890663
New Minimum: -0.7433133762811734 > -0.7560684950995572
WOLFE (weak): th(51.70643256076521)=-0.7560684950995572; dx=-3.967627754228107E-4 evalInputDelta=0.016950202065274422
New Minimum: -0.7560684950995572 > -0.8253942752985743
WOLFE (weak): th(258.53216280382605)=-0.8253942752985743; dx=-4.1740502180827923E-4 evalInputDelta=0.08627598226429156
New Minimum: -0.8253942752985743 > -1.3128702009643267
WOLFE (weak): th(1551.1929768229563)=-1.3128702009643267; dx=-3.983010322353316E-4 evalInputDelta=0.5737519079300439
New Minimum: -1.3128702009643267 > -2.6927399598660866
END: th(10858.350837760694)=-2.6927399598660866; dx=-4.695141309819564E-5 evalInputDelta=1.9536216668318038
Fitness changed from -0.7391182930342828 to -2.6927399598660866
Iteration 1 complete. Error: -2.6927399598660866 Total: 1.2816; Orientation: 0.0016; Line Search: 0.9322
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 6cdb371a to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-2.6927399598660866;dx=-6.90788546098404E-4
Armijo: th(23393.607721408407)=-1.96025333087221; dx=1.0483494895641953E-5 evalInputDelta=-0.7324866289938765
New Minimum: -2.6927399598660866 > -3.295788132878882
WOLF (strong): th(11696.803860704204)=-3.295788132878882; dx=6.510713985458855E-6 evalInputDelta=0.6030481730127955
New Minimum: -3.295788132878882 > -3.698464558907628
END: th(3898.9346202347347)=-3.698464558907628; dx=-2.643862021187307E-4 evalInputDelta=1.0057245990415415
Fitness changed from -2.6927399598660866 to -3.698464558907628
Iteration 2 complete. Error: -3.698464558907628 Total: 0.5940; Orientation: 0.0015; Line Search: 0.4703
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 2b10aa46 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-3.698464558907628;dx=-5.159225891277636E-4
New Minimum: -3.698464558907628 > -4.630160817243972
END: th(8400.000000000002)=-4.630160817243972; dx=-1.0685933538015458E-4 evalInputDelta=0.931696258336344
Fitness changed from -3.698464558907628 to -4.630160817243972
Iteration 3 complete. Error: -4.630160817243972 Total: 0.3634; Orientation: 0.0015; Line Search: 0.2377
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 341e606e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.061e+03, gradient 2.729e-02, dot -0.397; [988d6ce6-7014-40a6-8c89-a6ca6ff2bc72 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.630160817243972, -3.698464558907628, -2.6927399598660866, -0.7391182930342828
LBFGS Accumulation History: 3 points
Removed measurement 341e606e to history. Total: 3
th(0)=-4.630160817243972;dx=-7.23146901183747E-4
New Minimum: -4.630160817243972 > -5.69132162475319
END: th(18097.251396267828)=-5.69132162475319; dx=-1.0583663500170882E-4 evalInputDelta=1.0611608075092178
Fitness changed from -4.630160817243972 to -5.69132162475319
Iteration 4 complete. Error: -5.69132162475319 Total: 0.3904; Orientation: 0.0302; Line Search: 0.2365
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 64f73a61 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.436e+03, gradient 2.671e-02, dot -0.276; [988d6ce6-7014-40a6-8c89-a6ca6ff2bc72 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.69132162475319, -3.698464558907628, -2.6927399598660866, -0.7391182930342828
LBFGS Accumulation History: 3 points
Removed measurement 64f73a61 to history. Total: 3
th(0)=-5.69132162475319;dx=-6.372974791144792E-4
Armijo: th(38989.346202347355)=-4.084776139806965; dx=9.478313431750595E-6 evalInputDelta=-1.606545484946225
New Minimum: -5.69132162475319 > -6.388524049474474
END: th(19494.673101173677)=-6.388524049474474; dx=-5.2443690396552406E-5 evalInputDelta=0.6972024247212838
Fitness changed from -5.69132162475319 to -6.388524049474474
Iteration 5 complete. Error: -6.388524049474474 Total: 0.5004; Orientation: 0.0303; Line Search: 0.3465
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 4cdfa1a5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.210e+03, gradient 2.872e-02, dot -0.228; [988d6ce6-7014-40a6-8c89-a6ca6ff2bc72 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.388524049474474, -3.698464558907628, -2.6927399598660866, -0.7391182930342828
LBFGS Accumulation History: 3 points
Removed measurement 4cdfa1a5 to history. Total: 3
th(0)=-6.388524049474474;dx=-7.452101419869314E-4
Armijo: th(42000.000000000015)=-4.254097342146565; dx=2.840499295332229E-5 evalInputDelta=-2.1344267073279086
New Minimum: -6.388524049474474 > -8.5471989249725
WOLF (strong): th(21000.000000000007)=-8.5471989249725; dx=2.3692378602912516E-5 evalInputDelta=2.1586748754980265
New Minimum: -8.5471989249725 > -9.790755336335899
END: th(7000.000000000003)=-9.790755336335899; dx=-2.9913804720993367E-4 evalInputDelta=3.402231286861425
Fitness changed from -6.388524049474474 to -9.790755336335899
Iteration 6 complete. Error: -9.790755336335899 Total: 0.6067; Orientation: 0.0274; Line Search: 0.4557
Adding measurement 2ce0ca06 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.721e+03, gradient 2.355e-02, dot -0.265; [988d6ce6-7014-40a6-8c89-a6ca6ff2bc72 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -9.790755336335899, -3.698464558907628, -2.6927399598660866, -0.7391182930342828
LBFGS Accumulation History: 3 points
Removed measurement 2ce0ca06 to history. Total: 3
th(0)=-9.790755336335899;dx=-5.076168708534833E-4
New Minimum: -9.790755336335899 > -11.619389365833744
END: th(15081.042830223192)=-11.619389365833744; dx=-1.1772854165690627E-4 evalInputDelta=1.8286340294978451
Fitness changed from -9.790755336335899 to -11.619389365833744
Iteration 7 complete. Error: -11.619389365833744 Total: 0.3627; Orientation: 0.0284; Line Search: 0.2254
Adding measurement 55a047fd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.933e+03, gradient 2.506e-02, dot -0.252; [988d6ce6-7014-40a6-8c89-a6ca6ff2bc72 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.619389365833744, -3.698464558907628, -2.6927399598660866, -0.7391182930342828
LBFGS Accumulation History: 3 points
Removed measurement 55a047fd to history. Total: 3
th(0)=-11.619389365833744;dx=-6.029126849597828E-4
New Minimum: -11.619389365833744 > -11.667186317856055
WOLF (strong): th(32491.12183528947)=-11.667186317856055; dx=3.0834052687422604E-5 evalInputDelta=0.0477969520223116
New Minimum: -11.667186317856055 > -13.71099618906151
END: th(16245.560917644734)=-13.71099618906151; dx=-4.4947152745757356E-5 evalInputDelta=2.0916068232277656
Fitness changed from -11.619389365833744 to -13.71099618906151
Iteration 8 complete. Error: -13.71099618906151 Total: 0.4892; Orientation: 0.0267; Line Search: 0.3499
Adding measurement 2ea2dae3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.270e+04, gradient 2.420e-02, dot -0.264; [988d6ce6-7014-40a6-8c89-a6ca6ff2bc72 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -13.71099618906151, -3.698464558907628, -2.6927399598660866, -0.7391182930342828
LBFGS Accumulation History: 3 points
Removed measurement 2ea2dae3 to history. Total: 3
th(0)=-13.71099618906151;dx=-5.818106366574784E-4
Armijo: th(35000.00000000002)=-13.125309238890273; dx=7.07100646270089E-5 evalInputDelta=-0.5856869501712367
New Minimum: -13.71099618906151 > -16.240306407923452
END: th(17500.00000000001)=-16.240306407923452; dx=-2.5068251798092723E-5 evalInputDelta=2.5293102188619425
Fitness changed from -13.71099618906151 to -16.240306407923452
Iteration 9 complete. Error: -16.240306407923452 Total: 0.4951; Orientation: 0.0284; Line Search: 0.3502
Adding measurement 336915d1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.503e+03, gradient 2.522e-02, dot -0.223; [988d6ce6-7014-40a6-8c89-a6ca6ff2bc72 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.240306407923452, -3.698464558907628, -2.6927399598660866, -0.7391182930342828
LBFGS Accumulation History: 3 points
Removed measurement 336915d1 to history. Total: 3
th(0)=-16.240306407923452;dx=-5.787176241094409E-4
Armijo: th(37702.60707555799)=-14.164022460881615; dx=1.1500925644169753E-4 evalInputDelta=-2.076283947041837
New Minimum: -16.240306407923452 > -19.319267391836657
WOLF (strong): th(18851.303537778997)=-19.319267391836657; dx=1.674561760922388E-5 evalInputDelta=3.0789609839132055
END: th(6283.7678459263325)=-19.27744297745212; dx=-2.9317110014541133E-4 evalInputDelta=3.03713656952867
Fitness changed from -16.240306407923452 to -19.319267391836657
Iteration 10 complete. Error: -19.319267391836657 Total: 0.6276; Orientation: 0.0319; Line Search: 0.4818
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 73cde663 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.268e+04, gradient 2.673e-02, dot -0.227; [988d6ce6-7014-40a6-8c89-a6ca6ff2bc72 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.319267391836657, -3.698464558907628, -2.6927399598660866, -0.7391182930342828
LBFGS Accumulation History: 3 points
Removed measurement 73cde663 to history. Total: 3
th(0)=-19.319267391836657;dx=-6.790622402222485E-4
New Minimum: -19.319267391836657 > -22.9976896496943
END: th(13537.967431370616)=-22.9976896496943; dx=-9.813714916644828E-5 evalInputDelta=3.678422257857644
Fitness changed from -19.319267391836657 to -22.9976896496943
Iteration 11 complete. Error: -22.9976896496943 Total: 0.3977; Orientation: 0.0349; Line Search: 0.2347
Adding measurement 7c070d93 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.327e+04, gradient 2.133e-02, dot -0.280; [988d6ce6-7014-40a6-8c89-a6ca6ff2bc72 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -22.9976896496943, -3.698464558907628, -2.6927399598660866, -0.7391182930342828
LBFGS Accumulation History: 3 points
Removed measurement 7c070d93 to history. Total: 3
th(0)=-22.9976896496943;dx=-4.386261900933215E-4
New Minimum: -22.9976896496943 > -23.67917951945206
WOLF (strong): th(29166.666666666693)=-23.67917951945206; dx=2.047543782141955E-5 evalInputDelta=0.6814898697577583
New Minimum: -23.67917951945206 > -25.89941336640467
END: th(14583.333333333347)=-25.89941336640467; dx=-1.5665878168709955E-4 evalInputDelta=2.901723716710368
Fitness changed from -22.9976896496943 to -25.89941336640467
Iteration 12 complete. Error: -25.89941336640467 Total: 0.5067; Orientation: 0.0337; Line Search: 0.3545
Adding measurement 4d425dda to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.317e+08, gradient 2.246e-02, dot -0.272; [988d6ce6-7014-40a6-8c89-a6ca6ff2bc72 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -25.89941336640467, -3.698464558907628, -2.6927399598660866, -0.7391182930342828
LBFGS Accumulation History: 3 points
Removed measurement 4d425dda to history. Total: 3
th(0)=-25.89941336640467;dx=-4.6517856996341155E-4
New Minimum: -25.89941336640467 > -28.673916164689693
END: th(31418.839229631667)=-28.673916164689693; dx=-4.0063295449785336E-5 evalInputDelta=2.774502798285024
Fitness changed from -25.89941336640467 to -28.673916164689693
Iteration 13 complete. Error: -28.673916164689693 Total: 0.3810; Orientation: 0.0297; Line Search: 0.2367
Adding measurement 6e64befd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.449e+04, gradient 3.061e-02, dot -0.163; [988d6ce6-7014-40a6-8c89-a6ca6ff2bc72 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -28.673916164689693, -3.698464558907628, -2.6927399598660866, -0.7391182930342828
LBFGS Accumulation History: 3 points
Removed measurement 6e64befd to history. Total: 3
th(0)=-28.673916164689693;dx=-8.879001707862175E-4
Armijo: th(67689.83715685309)=-18.145687706099302; dx=1.057800917033362E-4 evalInputDelta=-10.528228458590391
New Minimum: -28.673916164689693 > -31.709598817886175
WOLF (strong): th(33844.91857842654)=-31.709598817886175; dx=5.635531836609253E-5 evalInputDelta=3.0356826531964813
New Minimum: -31.709598817886175 > -35.17011016797333
END: th(11281.63952614218)=-35.17011016797333; dx=-2.914263592726698E-4 evalInputDelta=6.496194003283634
Fitness changed from -28.673916164689693 to -35.17011016797333
Iteration 14 complete. Error: -35.17011016797333 Total: 0.5969; Orientation: 0.0289; Line Search: 0.4595
Adding measurement 41a95b07 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 2.509e+03, gradient 2.273e-02, dot 0.414; [988d6ce6-7014-40a6-8c89-a6ca6ff2bc72 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-35.17011016797333;dx=-23.351117148771493
New Minimum: -35.17011016797333 > -64.40418447240987
WOLF (strong): th(24305.555555555577)=-64.40418447240987; dx=6.71630789640453E-4 evalInputDelta=29.23407430443654
New Minimum: -64.40418447240987 > -64.4447014529243
WOLF (strong): th(12152.777777777788)=-64.4447014529243; dx=0.00131790162109613 evalInputDelta=29.27459128495098
WOLF (strong): th(4050.9259259259293)=-64.37853487947632; dx=0.0040679504612303186 evalInputDelta=29.208424711502992
WOLF (strong): th(1012.7314814814823)=-64.27007001459714; dx=0.015828800184193942 evalInputDelta=29.09995984662381
New Minimum: -64.4447014529243 > -65.91952450378324
WOLF (strong): th(202.54629629629648)=-65.91952450378324; dx=0.07477179613644834 evalInputDelta=30.74941433580991
New Minimum: -65.91952450378324 > -66.91964465518691
WOLF (strong): th(33.75771604938274)=-66.91964465518691; dx=0.2869618974466215 evalInputDelta=31.749534487213587
New Minimum: -66.91964465518691 > -73.53271295129625
END: th(4.822530864197534)=-73.53271295129625; dx=-2.040830985818431 evalInputDelta=38.36260278332292
Fitness changed from -35.17011016797333 to -73.53271295129625
Iteration 15 complete. Error: -73.53271295129625 Total: 1.1403; Orientation: 0.0299; Line Search: 0.9915
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -73.53271295129625 (> -Infinity) after 8.746s (< 5400.000s)
Returns
-73.53271295129625